AssertionError: assertion failed

Hail Batch & General Cloud
1

Hi!
I am trying to generate hail indexes for the imputed bgen files in RAP (dnanexus platform) for UK Biobank data processing. The command bellow has worked fine for all the chromosomes except chromosome 4, for which I am getting the error below. What might be the issue?

Thank you very much in advance!

file_url = f"file://{bgen_path}/ukb22828_c4_b0_v3.bgen"
hl.index_bgen(file_url,
index_file_map={file_url:f"hdfs:///ukb22828_c4_b0_v3.bgen.idx2"},
contig_recoding={“04”: “4”},
reference_genome=“GRCh37”,
skip_invalid_loci=True)

Traceback (most recent call last): (0 + 1) / 1]
File “”, line 1, in
File “”, line 2, in index_bgen
File “/opt/conda/lib/python3.9/site-packages/hail/typecheck/check.py”, line 584, in wrapper
return original_func(*args, **kwargs)
File “/opt/conda/lib/python3.9/site-packages/hail/methods/impex.py”, line 3014, in index_bgen
for r in ht.collect():
File “”, line 2, in collect
File “/opt/conda/lib/python3.9/site-packages/hail/typecheck/check.py”, line 584, in wrapper
return original_func(*args, **kwargs)
File “/opt/conda/lib/python3.9/site-packages/hail/table.py”, line 2178, in collect
return Env.backend().execute(e._ir, timed=_timed)
File “/opt/conda/lib/python3.9/site-packages/hail/backend/py4j_backend.py”, line 82, in execute
raise e.maybe_user_error(ir) from None
File “/opt/conda/lib/python3.9/site-packages/hail/backend/py4j_backend.py”, line 76, in execute
result_tuple = self._jbackend.executeEncode(jir, stream_codec, timed)
File “/cluster/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py”, line 1321, in call
File “/opt/conda/lib/python3.9/site-packages/hail/backend/py4j_backend.py”, line 35, in deco
raise fatal_error_from_java_error_triplet(deepest, full, error_id) from None
hail.utils.java.FatalError: AssertionError: assertion failed

Java stack trace:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 13.0 failed 4 times, most recent failure: Lost task 0.3 in stage 13.0 (TID 52) (ip-10-60-65-131.eu-west-2.compute.internal executor 0): java.lang.AssertionError: assertion failed

2

Hi @irun, I’m sorry Hail is failing for you. I expect this to work fine.

We nee the full and complete stack trace to diagnose. We need everything after the line “Java stack trace:”. Can you share all of that?

We will probably also need the Hail log file. You can find that in the working directory of your Python process or Jupyter Notebook.

We also need to know which version of Hail you are using. That should be included in the full and complete stack trace. If it is not, please add it!

One common cause of this error is trying to use an old version of Hail with BGEN v1.3 files. Support for BGEN v1.3 was added in Hail 0.2.108 which was released ten months ago. Due to the small size of our team and large number of users, we cannot support old versions of Hail. We’ll need you to update to the latest version, 0.2.126, if you’re not already on the latest version.

3

@irun I am facing the same issue, how did you navigate through it, My Hail version is 0.2.116, I am getting the error in the same file. Would appreciate any help