Hi Hail team,
I am running the following code
from gnomad.sample_qc.relatedness import filter_mt_to_trios
from gnomad.utils.filtering import filter_to_autosomes
from gnomad.variant_qc.pipeline import generate_trio_stats
from ukbb_qc.resources.basics import get_ukbb_data
from ukbb_qc.resources.sample_qc import (
array_concordance_results_path,
array_concordance_sites_path,
inferred_ped_path,
relatedness_ht_path,
)
from ukbb_qc.resources.variant_qc import var_annotations_ht_path
mt = get_ukbb_data(data_source, freeze)
ped_fp = inferred_ped_path(data_source, freeze)
ped = hl.Pedigree.read(ped_fp, delimiter="\t")
fam_ht = hl.import_fam(ped_fp, delimiter="\t")
mt = filter_to_autosomes(mt)
mt = filter_mt_to_trios(mt, fam_ht)
mt = hl.experimental.densify(mt)
mt = hl.trio_matrix(mt, pedigree=ped, complete_trios=True)
trio_stats_ht = generate_trio_stats(mt)
trio_stats_ht.naive_coalesce(n_partitions).write(
var_annotations_ht_path("trio_stats", data_source, freeze),
overwrite=True,
)
And I get a ArrayIndexOutOfBoundsException: 1 error. I am using hail 0.2.40 and the full log is attached.
I have previously run this code without error and at the time I was using this whl that Tim gave me:
https://storage.googleapis.com/hail-common/hailctl/dataproc/tpoterba-dev/0.2.33-d1759a52836e/hail-0.2.33-py3-none-any.whl
Thanks in advance for your help!
-Julia
Sorry you’re running into this problem. I don’t see where the log is attached though.
Yeah sorry I don’t think it finished uploading let me try again
Yes, that’s helpful, thanks. Investigating.
Alright, Arcturus looked into this, and this is a bug we introduced in 0.2.40. There’s a fix here: https://github.com/hail-is/hail/pull/8786 , will be included in 0.2.41.
Probably the easiest thing for you to do is to use 0.2.39 until 0.2.41 comes out. If there’s some reason you specifically need 0.2.40, we’ll get a custom wheel going.
Great, thank you! I just got it rerunning with 0.2.39, I will let you know how it goes
Seems to still be a problem with 0.2.39. So maybe something changed about the code and there is an error on my end. Here is the log from the run on 0.2.39.
https://drive.google.com/open?id=1cf3zM9_16xcIzHIY96nFGQFo_dBAIJan
This is actually a different, related problem. I’ll dig into it more tomorrow, but hopefully won’t be too hard to fix / get you a patched wheel.
That is great to hear, thank you!
Figured out the problem, working on a fix currently.
This should fix: https://github.com/hail-is/hail/pull/8799
Our plan is just to release 0.2.41 after it goes in since other people will likely hit this bug too.
Slightly more complicated than expected, but we will plan on releasing 0.2.41 today
Sorry to hear it’s more complicated, and thank you for working so hard on it!
Ok, hail version 0.2.41 has been released, this should hopefully fix your problem.
Thank you so much for how quickly you put this in! I will check it now.
Small test worked perfectly! So I can kick off the larger job
Great! Glad it worked out. Thanks for the bug reports.