# Hail Query & hailctl **URL:** https://discuss.hail.is/c/help-02/16.md [Latest](https://discuss.hail.is/latest.md) · [Categories](https://discuss.hail.is/categories.md) --- ## [About the Hail Query & hailctl category](https://discuss.hail.is/t/about-the-hail-query-hailctl-category/706)
**Author:** [@hail-team](https://discuss.hail.is/u/hail-team)\ **Replies:** 0
Help and discussion on using the Hail Query and hailctl. Do you have questions about how to best express your analysis in Hail? Are you perplexed by an error message or unexpected result? This is the place to get feedb… --- ## [Hail process seems unreasonably expensive](https://discuss.hail.is/t/hail-process-seems-unreasonably-expensive/4182)
**Author:** [@stephan](https://discuss.hail.is/u/stephan)\ **Replies:** 2\ **Last updated:** [September 28, 2026, 5:45pm UTC](https://discuss.hail.is/t/hail-process-seems-unreasonably-expensive/4182 "2026-09-28T17:45:12Z")
Hi all, Below I have pasted the code I want to use to subset the AllofUs Hail ACAF matrix for the set of SNPs that I want (7.7 million), as well as to repartition it for faster performance downstream. The matrix curren… --- ## [ZipException: File does not conform to block gzip format](https://discuss.hail.is/t/zipexception-file-does-not-conform-to-block-gzip-format/4257)
**Author:** [@najamulias](https://discuss.hail.is/u/najamulias)\ **Replies:** 0\ **Last updated:** [September 10, 2026, 10:16am UTC](https://discuss.hail.is/t/zipexception-file-does-not-conform-to-block-gzip-format/4257 "2026-09-10T10:16:08Z")
I am running a VariantDatasetCombiner with Hail version: Hail version: 0.2.137-1fc852d5db9a I am combining approximately 1,120 GVCF files stored in Google Cloud Storage. The combiner fails with the following error: E… --- ## [Duplicate sample IDs accepted silently by new\_combiner and what is the supported way to drop one column?](https://discuss.hail.is/t/duplicate-sample-ids-accepted-silently-by-new-combiner-and-what-is-the-supported-way-to-drop-one-column/4255)
**Author:** [@najamulias](https://discuss.hail.is/u/najamulias)\ **Replies:** 0\ **Last updated:** [August 18, 2026, 1:13am UTC](https://discuss.hail.is/t/duplicate-sample-ids-accepted-silently-by-new-combiner-and-what-is-the-supported-way-to-drop-one-column/4255 "2026-08-18T01:13:25Z")
Hi, Up front: this is not a bug report. The combiner did exactly what my inputs told it to do, and I have already traced the root cause on my side. What I am asking about is design and supported usage two questions: A… --- ## [Compatibility with VCFv4.5 spec](https://discuss.hail.is/t/compatibility-with-vcfv4-5-spec/4253)
**Author:** [@ERees](https://discuss.hail.is/u/ERees)\ **Replies:** 2\ **Last updated:** [July 9, 2026, 2:47pm UTC](https://discuss.hail.is/t/compatibility-with-vcfv4-5-spec/4253 "2026-07-09T14:47:16Z")
Hi all, I am trying to import a VCF with v4.5 spec, and getting the following error Hail version: 0.2.137-733ac4ccd943 Error summary: InvalidHeader: Your input file has a malformed header: VCFv4.5 is not a supported v… --- ## [FILTER field when using VDS Combiner](https://discuss.hail.is/t/filter-field-when-using-vds-combiner/4241)
**Author:** [@DBScan](https://discuss.hail.is/u/DBScan)\ **Replies:** 3\ **Last updated:** [June 1, 2026, 1:33pm UTC](https://discuss.hail.is/t/filter-field-when-using-vds-combiner/4241 "2026-06-01T13:33:46Z")
Hi, is it possible to get the FILTER field from a gVCF into the VDS? Usually I set the genotype to “missing”, if the FILTER field is not “PASS”. I haven’t found a way to specify that when running the VDS Combiner. --- ## [Large gVCF into VDS](https://discuss.hail.is/t/large-gvcf-into-vds/4103)
**Author:** [@mhebrard](https://discuss.hail.is/u/mhebrard)\ **Replies:** 22\ **Last updated:** [April 24, 2026, 12:36am UTC](https://discuss.hail.is/t/large-gvcf-into-vds/4103 "2026-04-24T00:36:35Z")
Hi team ! Asking for some insight / tips I have a large collection of gVCF files. Testing on 10,000 first with in view to scale up to 50,000 and 100,000 participants in due time. I wish to load the single sample gVCFs … --- ## [Hail on Databricks](https://discuss.hail.is/t/hail-on-databricks/4244)
**Author:** [@Elena](https://discuss.hail.is/u/Elena)\ **Replies:** 1\ **Last updated:** [March 11, 2026, 8:21pm UTC](https://discuss.hail.is/t/hail-on-databricks/4244 "2026-03-11T20:21:01Z")
Hello, The official documentation for installing Hail on Databricks appears to be outdated. The instructions on this page lead to links that return a 404 error: https://hail.is/docs/0.2/cloud/databricks.html#initialize… --- ## [Problems attaching reference sequence with add\_sequence() on UKB RAP](https://discuss.hail.is/t/problems-attaching-reference-sequence-with-add-sequence-on-ukb-rap/4243)
**Author:** [@jbs](https://discuss.hail.is/u/jbs)\ **Replies:** 1\ **Last updated:** [March 6, 2026, 6:13pm UTC](https://discuss.hail.is/t/problems-attaching-reference-sequence-with-add-sequence-on-ukb-rap/4243 "2026-03-06T18:13:24Z")
Hello! I am running into an odd issue with attaching GRCh38 fasta and fasta.fai files to a ReferenceGenome object. For context, this is running on UK Biobank RAP in Jupyter Lab with a Spark cluster. I am not sure if t… --- ## [Repartitioning after significant filtering](https://discuss.hail.is/t/repartitioning-after-significant-filtering/4235)
**Author:** [@grhodes26](https://discuss.hail.is/u/grhodes26)\ **Replies:** 1\ **Last updated:** [March 3, 2026, 5:36pm UTC](https://discuss.hail.is/t/repartitioning-after-significant-filtering/4235 "2026-03-03T17:36:55Z")
Hi! I’m using hail within a cloud environment to analyze srWGS data from All of Us and running into difficulty with having a large number of partitions after performing significant filtering. The initial matrix table ha… --- ## [LA entry is "NA" when samples is hom-ref in VDS](https://discuss.hail.is/t/la-entry-is-na-when-samples-is-hom-ref-in-vds/4242)
**Author:** [@DBScan](https://discuss.hail.is/u/DBScan)\ **Replies:** 0\ **Last updated:** [February 25, 2026, 3:11pm UTC](https://discuss.hail.is/t/la-entry-is-na-when-samples-is-hom-ref-in-vds/4242 "2026-02-25T15:11:28Z")
Hi, I’m a bit struggling with formatting my VDS file from local to global. Currently I have to following code to transform my local entries LAD and LGT into their global equivalent: mt = vds.variant\_data.annotate\_entri… --- ## [Spark–Hail failure mode triggered by .write() on large MatrixTables](https://discuss.hail.is/t/spark-hail-failure-mode-triggered-by-write-on-large-matrixtables/4229)
**Author:** [@nlbxuyen](https://discuss.hail.is/u/nlbxuyen)\ **Replies:** 4\ **Last updated:** [January 20, 2026, 5:57pm UTC](https://discuss.hail.is/t/spark-hail-failure-mode-triggered-by-write-on-large-matrixtables/4229 "2026-01-20T17:57:22Z")
Hello, I am trying to extract variants (universe) from UKB DRAGEN WGS pVCF 500k in all autosomes. First I have tried chromosome 21. I used Python notebook in Spark Cluster, feature HAIL with mem1\_ssd1\_v2\_x16 with 50 ins… --- ## [GWAS add genotype as covariate](https://discuss.hail.is/t/gwas-add-genotype-as-covariate/4219)
**Author:** [@Luis](https://discuss.hail.is/u/Luis)\ **Replies:** 1\ **Last updated:** [January 5, 2026, 2:50pm UTC](https://discuss.hail.is/t/gwas-add-genotype-as-covariate/4219 "2026-01-05T14:50:15Z")
Hello, Can we add an additional genotype as a covariate when we run GWAS with HAIL? Thanks in advance! --- ## [Error when using the write() function](https://discuss.hail.is/t/error-when-using-the-write-function/4215)
**Author:** [@marise](https://discuss.hail.is/u/marise)\ **Replies:** 1\ **Last updated:** [November 18, 2025, 5:51pm UTC](https://discuss.hail.is/t/error-when-using-the-write-function/4215 "2025-11-18T17:51:07Z")
Hello i am doing the following code from pyspark.sql import SparkSession import hail as hl builder = ( SparkSession .builder .enableHiveSupport() ) spark = builder.getOrCreate() hl.init(sc=spark.sparkContext,dr… --- ## [Partitioning and Performance of VDS Methods](https://discuss.hail.is/t/partitioning-and-performance-of-vds-methods/4199)
**Author:** [@brand](https://discuss.hail.is/u/brand)\ **Replies:** 2\ **Last updated:** [October 30, 2025, 10:19am UTC](https://discuss.hail.is/t/partitioning-and-performance-of-vds-methods/4199 "2025-10-30T10:19:52Z")
Dear Hail Team, iam currently executing a large-ish hail pipeline on an on-premises spark cluster on different local HPC systems and observed some behaviour that I am struggling to understand relating to the resource us… --- ## [Snp\_matrix \<- fread("File.raw") Out of Memory](https://discuss.hail.is/t/snp-matrix-fread-file-raw-out-of-memory/4202)
**Author:** [@HugHo](https://discuss.hail.is/u/HugHo)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:54pm UTC](https://discuss.hail.is/t/snp-matrix-fread-file-raw-out-of-memory/4202 "2025-10-29T19:54:54Z")
Hello, I would like to perform a GWAS and I already have the .map and raw. files with the phenotype file but the raw file weight 4Go ( output ls -l on puhti server -rw-rw---- 1 huguetho huguetho 4004201413 Aug 4 13:10 … --- ## [Hail not working with recent versions of VEP](https://discuss.hail.is/t/hail-not-working-with-recent-versions-of-vep/4204)
**Author:** [@sk4](https://discuss.hail.is/u/sk4)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:41pm UTC](https://discuss.hail.is/t/hail-not-working-with-recent-versions-of-vep/4204 "2025-10-29T19:41:42Z")
Hi! I have Hail and VEP installed as modules on an HPC cluster, and I am trying to perform functional annotation on an imported VCF using VEP through Hail in order to distribute the task across multiple nodes. However, … --- ## [HailException: array index out of bounds when converting Hail Table to Pandas DataFrame](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193)
**Author:** [@hrithikgupta88](https://discuss.hail.is/u/hrithikgupta88)\ **Replies:** 2\ **Last updated:** [October 29, 2025, 7:32pm UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193 "2025-10-29T19:32:53Z")
Description: We are processing VCF files stored in S3 using Hail in Python, and encountering the following error when calling .to\_pandas() on a selected Hail Table: hail.utils.java.HailException: array index out of bou… --- ## [Does liftover update alleles](https://discuss.hail.is/t/does-liftover-update-alleles/4205)
**Author:** [@hyslin](https://discuss.hail.is/u/hyslin)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:08pm UTC](https://discuss.hail.is/t/does-liftover-update-alleles/4205 "2025-10-29T19:08:16Z")
When I applied liftover from GRCh38 to GRCh37 on my Hail table, the process was pretty fast but it didn’t seem to update the alleles. Does liftover also change the alleles? If not, what is the recommended way to update t… --- ## [Merging new genotype fields](https://discuss.hail.is/t/merging-new-genotype-fields/4208)
**Author:** [@handecadir](https://discuss.hail.is/u/handecadir)\ **Replies:** 0\ **Last updated:** [October 23, 2025, 3:02pm UTC](https://discuss.hail.is/t/merging-new-genotype-fields/4208 "2025-10-23T15:02:07Z")
Dear Hail community, We have a matrix table we have created from 92 cases and 100 controls. These were not joint variant called together. Cases were joint called and controls were also joint called seperately. We have … --- ## [Convert Clair3 gVCF to VDS failed with NumberFormatException error](https://discuss.hail.is/t/convert-clair3-gvcf-to-vds-failed-with-numberformatexception-error/4207)
**Author:** [@Sol\_Li](https://discuss.hail.is/u/Sol_Li)\ **Replies:** 3\ **Last updated:** [October 7, 2025, 2:07pm UTC](https://discuss.hail.is/t/convert-clair3-gvcf-to-vds-failed-with-numberformatexception-error/4207 "2025-10-07T14:07:04Z")
Hi, Hope to get some insight into the issue I had above and thanks in advanced. I was trying to import 2 gVCF files named “sample1.wf\_snp.gvcf.gz” and “sample2.wf\_snp.gvcf.gz” intp hail and converting them into VDS wit… --- ## [Summarize\_variants divide by zero error](https://discuss.hail.is/t/summarize-variants-divide-by-zero-error/4200)
**Author:** [@jjfarrell](https://discuss.hail.is/u/jjfarrell)\ **Replies:** 1\ **Last updated:** [September 4, 2025, 1:15pm UTC](https://discuss.hail.is/t/summarize-variants-divide-by-zero-error/4200 "2025-09-04T13:15:12Z")
Using version 0.2.132-678e1f52b999…. sum\_all=hl.summarize\_variants(mt,show=False) This line is generating an error when dividing by 0 when calculating the Ti/Tv ratio. The matrix table contains only structural variant… --- ## [Drop unnecessary QC metrics](https://discuss.hail.is/t/drop-unnecessary-qc-metrics/4194)
**Author:** [@hyslin](https://discuss.hail.is/u/hyslin)\ **Replies:** 3\ **Last updated:** [August 13, 2025, 8:33pm UTC](https://discuss.hail.is/t/drop-unnecessary-qc-metrics/4194 "2025-08-13T20:33:47Z")
Hi, I’m performing variant QC on a MatrixTable with dimensions (56993186, 414830). The data has already been filtered to include only common variants. The remaining steps are to filter for missingness and retain only bi… --- ## [Local\_to\_global error when annotating LA from DRAGEN joint VCF: allele index out of bounds](https://discuss.hail.is/t/local-to-global-error-when-annotating-la-from-dragen-joint-vcf-allele-index-out-of-bounds/4172)
**Author:** [@inninun](https://discuss.hail.is/u/inninun)\ **Replies:** 2\ **Last updated:** [July 9, 2025, 6:08am UTC](https://discuss.hail.is/t/local-to-global-error-when-annotating-la-from-dragen-joint-vcf-allele-index-out-of-bounds/4172 "2025-07-09T06:08:26Z")
Hi Hail team, Thanks as always for the great tool and support! I’m currently working with a joint VCF generated from the DRAGEN pipeline and performing genotype QC using Hail (v0.2.135). When trying to annotate the AD f… --- ## [Combine non human gvcf with new\_combiner reports errors](https://discuss.hail.is/t/combine-non-human-gvcf-with-new-combiner-reports-errors/4171)
**Author:** [@atcg1122](https://discuss.hail.is/u/atcg1122)\ **Replies:** 1\ **Last updated:** [July 8, 2025, 4:03pm UTC](https://discuss.hail.is/t/combine-non-human-gvcf-with-new-combiner-reports-errors/4171 "2025-07-08T16:03:20Z")
i have thousends of rice gvcf, and i tryed to combine them with hail’s new\_combiner function, but report errors. ValueError: Unsupported reference genome 'IRGSP\_1\_0', only 'GRCh37' and 'GRCh38' are supported can new\_co… --- ## [Most severe VEP consequence](https://discuss.hail.is/t/most-severe-vep-consequence/4169)
**Author:** [@johnnyr](https://discuss.hail.is/u/johnnyr)\ **Replies:** 5\ **Last updated:** [July 8, 2025, 2:10pm UTC](https://discuss.hail.is/t/most-severe-vep-consequence/4169 "2025-07-08T14:10:46Z")
I have annotated a VCF with VEP outside of hail and have used ‘annotate\_rows’ in hail to get the desired annotation. So far so good, but due to this, I cannot use e.g. gnomad.utils (or at least don’t know how) to easily… --- ## [Illegal reflective access](https://discuss.hail.is/t/illegal-reflective-access/3897)
**Author:** [@anh151](https://discuss.hail.is/u/anh151)\ **Replies:** 1\ **Last updated:** [June 25, 2025, 3:33pm UTC](https://discuss.hail.is/t/illegal-reflective-access/3897 "2025-06-25T15:33:35Z")
Hello, version 0.2.130-bea04d9c79b5 within All of Us. I have a piece of hail code that produces this warning. Everything still works fine, but I wanted to report it just incase it’s an issue. I realize that it says to … --- ## [Connection rejection error when files are processed](https://discuss.hail.is/t/connection-rejection-error-when-files-are-processed/4165)
**Author:** [@Swati\_Srivastava](https://discuss.hail.is/u/Swati_Srivastava)\ **Replies:** 0\ **Last updated:** [June 23, 2025, 4:16pm UTC](https://discuss.hail.is/t/connection-rejection-error-when-files-are-processed/4165 "2025-06-23T16:16:26Z")
I am processing my csv batch files. (These are 22 files). It starts processing but then it give me this error: This is snippet of my code: — BATCH PROCESSING — for i in range(0, len(csv\_files), batch\_size): print(f"\\… --- ## [QUAL field during g.vcf combining](https://discuss.hail.is/t/qual-field-during-g-vcf-combining/4050)
**Author:** [@P\_W](https://discuss.hail.is/u/P_W)\ **Replies:** 2\ **Last updated:** [June 12, 2025, 3:49pm UTC](https://discuss.hail.is/t/qual-field-during-g-vcf-combining/4050 "2025-06-12T15:49:46Z")
Hi, I have simple question: Is it possible to force the QUAL field from a gvcf file to be loaded by vds.combiner? This value would be useful for calculating various fields required by VQSR (at later stages of analysis -… --- ## [Hail gVCF bgzip files are getting sorted?](https://discuss.hail.is/t/hail-gvcf-bgzip-files-are-getting-sorted/4156)
**Author:** [@Vipul\_Patel](https://discuss.hail.is/u/Vipul_Patel)\ **Replies:** 8\ **Last updated:** [June 11, 2025, 6:15pm UTC](https://discuss.hail.is/t/hail-gvcf-bgzip-files-are-getting-sorted/4156 "2025-06-11T18:15:06Z")
Hi I am currently playing around with hail and read up quite a bit of here and there to make it runnable, solving issues from not finding the file or how to create the storing the result. The issue, which I face now, … [Next page](https://discuss.hail.is/c/help-02/16.md?page=1)