# PL of haploid call induced error when split\_multi\_hts(), in VCF produced by Dragen pipeline **URL:** \ **Category:** Hail Query & hailctl\ **Created:** [February 9, 2023, 3:31pm UTC](https://discuss.hail.is/t/pl-of-haploid-call-induced-error-when-split-multi-hts-in-vcf-produced-by-dragen-pipeline/3116 "2023-02-09T15:31:11Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Leehyeji789](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/leehyeji789/32/899_2.png) [@Leehyeji789](https://discuss.hail.is/u/Leehyeji789)\ **Post date:** [February 9, 2023, 3:31pm UTC](https://discuss.hail.is/t/pl-of-haploid-call-induced-error-when-split-multi-hts-in-vcf-produced-by-dragen-pipeline/3116/1 "2023-02-09T15:31:11Z")
Hi All, I’m processing VCF produced by Dragen pipeline with Hail. When using split\_multi\_hts() to split multi-allelic site variants, I get the error of index out of bounds like below. ```python HailUserError: Error summary: HailException: array index out of bounds: index=10, length=7 ------------ Hail stack trace: File "", line 2, in mt = hl.split_multi_hts(mt) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2168, in split_multi_hts (hl.range(0, 3).map(lambda i: File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2172, in ).map(lambda j: split.PL[j])))))) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2172, in ).map(lambda j: split.PL[j])))))) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/expr/expressions/typed_expressions.py", line 481, in __getitem__ return self._method("indexArray", self.dtype.element_type, item) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/expr/expressions/base_expression.py", line 695, in _method x = ir.Apply(name, ret_type, self._ir, *(a._ir for a in args)) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/ir/ir.py", line 2628, in __init__ self.save_error_info() ``` I saw the similar case like me ([Error index out of bounds](https://discuss.hail.is/t/error-index-out-of-bounds/1766)). In this query, above error is due to PL fields. So I filtered calls which length of PL is 7 with following code, as the error message indicates. `tmp.filter_entries(tmp.PL.length() == 7).entries().show()` And I found those calls were from the sex chromosome of male samples, so they were all haploid calls. In my VCF, haploid calls are presented as haploid genotypes. This could make trouble in split\_multi\_hts() function because the function assumes that all calls are presented as diploidy. I’ll have to look for more, but Dragen pipeline basically represent haploid calls this way when producing VCF, unlike GATK. In line with this, impute\_sex() function also do not work. I think this could be a problem for people who use Hail to process Dragen-produced VCF. So here is my question. Is there a way to change haploid genotypes to diploid? Or any way to handle non-diploid calls when applying Hail functions? ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [February 9, 2023, 7:36pm UTC](https://discuss.hail.is/t/pl-of-haploid-call-induced-error-when-split-multi-hts-in-vcf-produced-by-dragen-pipeline/3116/2 "2023-02-09T19:36:14Z")
Hey @Leehyeji789 ! Sorry you’re running into this. I’ll look into the possibility of improving Hail to handle mixed diploid-haploid datasets. In the meantime, see below. * * * EDIT: I chatted with some analysts here. Until we have bandwidth to fix Hail’s methods to support haploids, your best bet is to recode to diploid: ```python mt = mt.annotate_entries( GT = hl.if_else( mt.GT.ploidy == 1, hl.call(mt.GT[0], mt.GT[0]), mt.GT) ) ``` * * * Can you share the script you executed? That will help us understand exactly what the issues are. I think the simplest thing for you to do is to analyze the haploid Y samples separately. Concretely, ```python mt = hl.read_matrix_table(...) mt = mt.filter_rows(mt.locus.contig != 'chrY') # or 'Y' depending on your reference ``` And then just do your analysis on diploids. Separately, you can analyze the Y chromosome: ```python mt = hl.read_matrix_table(...) mt = mt.filter_rows(mt.locus.contig == 'chrY') ``` ---
**Author:** ![Leehyeji789](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/leehyeji789/32/899_2.png) [@Leehyeji789](https://discuss.hail.is/u/Leehyeji789)\ **Post date:** [February 10, 2023, 2:15am UTC](https://discuss.hail.is/t/pl-of-haploid-call-induced-error-when-split-multi-hts-in-vcf-produced-by-dragen-pipeline/3116/3 "2023-02-10T02:15:54Z")
Thank you for answer! I will apply the alternative way you suggest. Here is the code I executed, and entire log file. [dragen\_test\_vcf\_log.txt](https://discuss.hail.is/uploads/short-url/8qk1H3iIuvSuZsRjy8IIdLxJP8O.txt) (1.6 MB) ```python mt = hl.import_vcf("dragen-joint-test.vcf.gz", reference_genome='GRCh38', force_bgz=True, skip_invalid_loci=True, n_partitions=144) d = hl.split_multi_hts(mt) d.write("dragne-wgs-test.mt", overwrite=True) -------------------------------------------------------------------------- HailUserError Traceback (most recent call last) in 1 mt = hl.split_multi_hts(mt) ----> 2 mt.write("dragne-wgs-test.mt", overwrite=True) in write(self, output, overwrite, stage_locally, _codec_spec, _partitions, _checkpoint_file) ~/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs) 575 def wrapper(__original_func, *args, **kwargs): 576 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method) --> 577 return __original_func(*args_, **kwargs_) 578 579 return wrapper ~/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/matrixtable.py in write(self, output, overwrite, stage_locally, _codec_spec, _partitions, _checkpoint_file) 2556 2557 writer = ir.MatrixNativeWriter(output, overwrite, stage_locally, _codec_spec, _partitions, _partitions_type, _checkpoint_file) -> 2558 Env.backend().execute(ir.MatrixWrite(self._mir, writer)) 2559 2560 class _Show: ~/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/backend/py4j_backend.py in execute(self, ir, timed) 102 return (value, timings) if timed else value 103 except FatalError as e: --> 104 self._handle_fatal_error_from_backend(e, ir) 105 106 async def _async_execute(self, ir, timed=False): ~/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/backend/backend.py in _handle_fatal_error_from_backend(self, err, ir) 187 'Hail stack trace:\n' 188 f'{better_stack_trace}') --> 189 raise HailUserError(message_and_trace) from None HailUserError: Error summary: HailException: array index out of bounds: index=2, length=2 ------------ Hail stack trace: File "", line 2, in mt = hl.split_multi_hts(mt) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2168, in split_multi_hts (hl.range(0, 3).map(lambda i: File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2172, in ).map(lambda j: split.PL[j])))))) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/methods/statgen.py", line 2172, in ).map(lambda j: split.PL[j])))))) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/expr/expressions/typed_expressions.py", line 481, in __getitem__ return self._method("indexArray", self.dtype.element_type, item) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/expr/expressions/base_expression.py", line 695, in _method x = ir.Apply(name, ret_type, self._ir, *(a._ir for a in args)) File "/home/sonic/bin/anaconda/envs/hail/lib/python3.7/site-packages/hail/ir/ir.py", line 2628, in __init__ self.save_error_info() ```