# Latest **URL:** https://discuss.hail.is/latest.md [Latest](https://discuss.hail.is/latest.md) · [Categories](https://discuss.hail.is/categories.md) --- ## [Welcome to the Hail community!](https://discuss.hail.is/t/welcome-to-the-hail-community/8)
**Author:** [@system](https://discuss.hail.is/u/system)\ **Replies:** 0\ **Last updated:** [August 26, 2016, 7:39pm UTC](https://discuss.hail.is/t/welcome-to-the-hail-community/8 "2016-08-26T19:39:27Z")
Welcome to the discussion forum for Hail, an open-source, scalable framework for exploring and analyzing genetic data. The Hail team is based in the Neale Lab at the Stanley Center for Psychiatric Research of the Broad … --- ## [Reducing serialization size on high-partition jobs](https://discuss.hail.is/t/reducing-serialization-size-on-high-partition-jobs/4254)
**Author:** [@Thouis](https://discuss.hail.is/u/Thouis)\ **Replies:** 2\ **Last updated:** [September 11, 2026, 11:28am UTC](https://discuss.hail.is/t/reducing-serialization-size-on-high-partition-jobs/4254 "2026-09-11T11:28:46Z")
Hi all, I’ve been working on some code for computing PRSes in AllOfUs, and was hitting warnings like: WARN TaskSetManager: Stage 2 contains a task of very large size (1528 KiB). The maximum recommended task size is 100… --- ## [ZipException: File does not conform to block gzip format](https://discuss.hail.is/t/zipexception-file-does-not-conform-to-block-gzip-format/4257)
**Author:** [@najamulias](https://discuss.hail.is/u/najamulias)\ **Replies:** 0\ **Last updated:** [September 10, 2026, 10:16am UTC](https://discuss.hail.is/t/zipexception-file-does-not-conform-to-block-gzip-format/4257 "2026-09-10T10:16:08Z")
I am running a VariantDatasetCombiner with Hail version: Hail version: 0.2.137-1fc852d5db9a I am combining approximately 1,120 GVCF files stored in Google Cloud Storage. The combiner fails with the following error: E… --- ## [Duplicate sample IDs accepted silently by new\_combiner and what is the supported way to drop one column?](https://discuss.hail.is/t/duplicate-sample-ids-accepted-silently-by-new-combiner-and-what-is-the-supported-way-to-drop-one-column/4255)
**Author:** [@najamulias](https://discuss.hail.is/u/najamulias)\ **Replies:** 0\ **Last updated:** [August 18, 2026, 1:13am UTC](https://discuss.hail.is/t/duplicate-sample-ids-accepted-silently-by-new-combiner-and-what-is-the-supported-way-to-drop-one-column/4255 "2026-08-18T01:13:25Z")
Hi, Up front: this is not a bug report. The combiner did exactly what my inputs told it to do, and I have already traced the root cause on my side. What I am asking about is design and supported usage two questions: A… --- ## [Compatibility with VCFv4.5 spec](https://discuss.hail.is/t/compatibility-with-vcfv4-5-spec/4253)
**Author:** [@ERees](https://discuss.hail.is/u/ERees)\ **Replies:** 2\ **Last updated:** [July 9, 2026, 2:47pm UTC](https://discuss.hail.is/t/compatibility-with-vcfv4-5-spec/4253 "2026-07-09T14:47:16Z")
Hi all, I am trying to import a VCF with v4.5 spec, and getting the following error Hail version: 0.2.137-733ac4ccd943 Error summary: InvalidHeader: Your input file has a malformed header: VCFv4.5 is not a supported v… --- ## [SNV De novo calling tool for cohort research at scale](https://discuss.hail.is/t/snv-de-novo-calling-tool-for-cohort-research-at-scale/4251)
**Author:** [@Eliawad14](https://discuss.hail.is/u/Eliawad14)\ **Replies:** 0\ **Last updated:** [June 2, 2026, 3:55pm UTC](https://discuss.hail.is/t/snv-de-novo-calling-tool-for-cohort-research-at-scale/4251 "2026-06-02T15:55:38Z")
My name is Elias and I work in the Genetics Department at Baylor College of Medicine. I created an add on function to the Hail program that can process WGS/WES VCF files for De novo variants in a build-agnostic, pedig… --- ## [FILTER field when using VDS Combiner](https://discuss.hail.is/t/filter-field-when-using-vds-combiner/4241)
**Author:** [@DBScan](https://discuss.hail.is/u/DBScan)\ **Replies:** 3\ **Last updated:** [June 1, 2026, 1:33pm UTC](https://discuss.hail.is/t/filter-field-when-using-vds-combiner/4241 "2026-06-01T13:33:46Z")
Hi, is it possible to get the FILTER field from a gVCF into the VDS? Usually I set the genotype to “missing”, if the FILTER field is not “PASS”. I haven’t found a way to specify that when running the VDS Combiner. --- ## [Large gVCF into VDS](https://discuss.hail.is/t/large-gvcf-into-vds/4103)
**Author:** [@mhebrard](https://discuss.hail.is/u/mhebrard)\ **Replies:** 22\ **Last updated:** [April 24, 2026, 12:36am UTC](https://discuss.hail.is/t/large-gvcf-into-vds/4103 "2026-04-24T00:36:35Z")
Hi team ! Asking for some insight / tips I have a large collection of gVCF files. Testing on 10,000 first with in view to scale up to 50,000 and 100,000 participants in due time. I wish to load the single sample gVCFs … --- ## [Hail on Databricks](https://discuss.hail.is/t/hail-on-databricks/4244)
**Author:** [@Elena](https://discuss.hail.is/u/Elena)\ **Replies:** 1\ **Last updated:** [March 11, 2026, 8:21pm UTC](https://discuss.hail.is/t/hail-on-databricks/4244 "2026-03-11T20:21:01Z")
Hello, The official documentation for installing Hail on Databricks appears to be outdated. The instructions on this page lead to links that return a 404 error: https://hail.is/docs/0.2/cloud/databricks.html#initialize… --- ## [ABFS (abfs://) paths with ADLS Gen2 no longer work in Hail 0.2.137 due to filesystem validation – is it expected / supported?](https://discuss.hail.is/t/abfs-abfs-paths-with-adls-gen2-no-longer-work-in-hail-0-2-137-due-to-filesystem-validation-is-it-expected-supported/4221)
**Author:** [@souckmi](https://discuss.hail.is/u/souckmi)\ **Replies:** 1\ **Last updated:** [March 11, 2026, 8:18pm UTC](https://discuss.hail.is/t/abfs-abfs-paths-with-adls-gen2-no-longer-work-in-hail-0-2-137-due-to-filesystem-validation-is-it-expected-supported/4221 "2026-03-11T20:18:02Z")
Hi, we recently upgraded our pipeline from an older Hail and ran into an issue when working with Azure Data Lake Storage Gen2 using ABFS (abfs://) paths. Our setup: Hail: 0.2.137 Spark: 3.5.7 Scala: 2.12 Java: 11 Pyt… --- ## [Problems attaching reference sequence with add\_sequence() on UKB RAP](https://discuss.hail.is/t/problems-attaching-reference-sequence-with-add-sequence-on-ukb-rap/4243)
**Author:** [@jbs](https://discuss.hail.is/u/jbs)\ **Replies:** 1\ **Last updated:** [March 6, 2026, 6:13pm UTC](https://discuss.hail.is/t/problems-attaching-reference-sequence-with-add-sequence-on-ukb-rap/4243 "2026-03-06T18:13:24Z")
Hello! I am running into an odd issue with attaching GRCh38 fasta and fasta.fai files to a ReferenceGenome object. For context, this is running on UK Biobank RAP in Jupyter Lab with a Spark cluster. I am not sure if t… --- ## [Repartitioning after significant filtering](https://discuss.hail.is/t/repartitioning-after-significant-filtering/4235)
**Author:** [@grhodes26](https://discuss.hail.is/u/grhodes26)\ **Replies:** 1\ **Last updated:** [March 3, 2026, 5:36pm UTC](https://discuss.hail.is/t/repartitioning-after-significant-filtering/4235 "2026-03-03T17:36:55Z")
Hi! I’m using hail within a cloud environment to analyze srWGS data from All of Us and running into difficulty with having a large number of partitions after performing significant filtering. The initial matrix table ha… --- ## [LA entry is "NA" when samples is hom-ref in VDS](https://discuss.hail.is/t/la-entry-is-na-when-samples-is-hom-ref-in-vds/4242)
**Author:** [@DBScan](https://discuss.hail.is/u/DBScan)\ **Replies:** 0\ **Last updated:** [February 25, 2026, 3:11pm UTC](https://discuss.hail.is/t/la-entry-is-na-when-samples-is-hom-ref-in-vds/4242 "2026-02-25T15:11:28Z")
Hi, I’m a bit struggling with formatting my VDS file from local to global. Currently I have to following code to transform my local entries LAD and LGT into their global equivalent: mt = vds.variant\_data.annotate\_entri… --- ## [Spark–Hail failure mode triggered by .write() on large MatrixTables](https://discuss.hail.is/t/spark-hail-failure-mode-triggered-by-write-on-large-matrixtables/4229)
**Author:** [@nlbxuyen](https://discuss.hail.is/u/nlbxuyen)\ **Replies:** 4\ **Last updated:** [January 20, 2026, 5:57pm UTC](https://discuss.hail.is/t/spark-hail-failure-mode-triggered-by-write-on-large-matrixtables/4229 "2026-01-20T17:57:22Z")
Hello, I am trying to extract variants (universe) from UKB DRAGEN WGS pVCF 500k in all autosomes. First I have tried chromosome 21. I used Python notebook in Spark Cluster, feature HAIL with mem1\_ssd1\_v2\_x16 with 50 ins… --- ## [GWAS add genotype as covariate](https://discuss.hail.is/t/gwas-add-genotype-as-covariate/4219)
**Author:** [@Luis](https://discuss.hail.is/u/Luis)\ **Replies:** 1\ **Last updated:** [January 5, 2026, 2:50pm UTC](https://discuss.hail.is/t/gwas-add-genotype-as-covariate/4219 "2026-01-05T14:50:15Z")
Hello, Can we add an additional genotype as a covariate when we run GWAS with HAIL? Thanks in advance! --- ## [Symbolic Alleles and Overlapping Variants](https://discuss.hail.is/t/symbolic-alleles-and-overlapping-variants/4214)
**Author:** [@Mireia](https://discuss.hail.is/u/Mireia)\ **Replies:** 2\ **Last updated:** [December 12, 2025, 9:11am UTC](https://discuss.hail.is/t/symbolic-alleles-and-overlapping-variants/4214 "2025-12-12T09:11:26Z")
Hello! I’ve developed a pipeline using hail that runs QC and calculates stratified AFs fields by sex and ancestry. While working with hl.split\_multi\_hts() and symbolic alleles (\*), I noticed that genotypes are not updat… --- ## [Writing MatrixTable on DNAnexus extremely slow + “server connection failed” at the end](https://discuss.hail.is/t/writing-matrixtable-on-dnanexus-extremely-slow-server-connection-failed-at-the-end/4218)
**Author:** [@marise](https://discuss.hail.is/u/marise)\ **Replies:** 0\ **Last updated:** [November 26, 2025, 2:14pm UTC](https://discuss.hail.is/t/writing-matrixtable-on-dnanexus-extremely-slow-server-connection-failed-at-the-end/4218 "2025-11-26T14:14:32Z")
Hi everyone, I’m working with UK Biobank exome sequences (450k individuals) on DNAnexus using Hail, and I’m having persistent issues when trying to write a MatrixTable. My goal is to create the MatrixTable so I can lat… --- ## [Error when using the write() function](https://discuss.hail.is/t/error-when-using-the-write-function/4215)
**Author:** [@marise](https://discuss.hail.is/u/marise)\ **Replies:** 1\ **Last updated:** [November 18, 2025, 5:51pm UTC](https://discuss.hail.is/t/error-when-using-the-write-function/4215 "2025-11-18T17:51:07Z")
Hello i am doing the following code from pyspark.sql import SparkSession import hail as hl builder = ( SparkSession .builder .enableHiveSupport() ) spark = builder.getOrCreate() hl.init(sc=spark.sparkContext,dr… --- ## [Using GPUs from HAIL](https://discuss.hail.is/t/using-gpus-from-hail/4216)
**Author:** [@jjfarrell](https://discuss.hail.is/u/jjfarrell)\ **Replies:** 1\ **Last updated:** [November 17, 2025, 4:47pm UTC](https://discuss.hail.is/t/using-gpus-from-hail/4216 "2025-11-17T16:47:28Z")
Is it possble to enable the use of GPUs with Hail through the RAPIDS Accelerator for Apache Spark? --- ## [Reproduce hl.pc\_project output using Plink](https://discuss.hail.is/t/reproduce-hl-pc-project-output-using-plink/4190)
**Author:** [@Hatoon](https://discuss.hail.is/u/Hatoon)\ **Replies:** 2\ **Last updated:** [October 31, 2025, 7:49pm UTC](https://discuss.hail.is/t/reproduce-hl-pc-project-output-using-plink/4190 "2025-10-31T19:49:05Z")
Hi! I am trying to reproduce Hail’s pc\_project output using plink –score. I am adding the variance-standardize option and keeping mean imputation. Plink notes that “PCs will be scaled a bit differently from ref\_data.eig… --- ## [Partitioning and Performance of VDS Methods](https://discuss.hail.is/t/partitioning-and-performance-of-vds-methods/4199)
**Author:** [@brand](https://discuss.hail.is/u/brand)\ **Replies:** 2\ **Last updated:** [October 30, 2025, 10:19am UTC](https://discuss.hail.is/t/partitioning-and-performance-of-vds-methods/4199 "2025-10-30T10:19:52Z")
Dear Hail Team, iam currently executing a large-ish hail pipeline on an on-premises spark cluster on different local HPC systems and observed some behaviour that I am struggling to understand relating to the resource us… --- ## [Error running hail 0.2.134-py3.11](https://discuss.hail.is/t/error-running-hail-0-2-134-py3-11/4201)
**Author:** [@Elizabeth\_Kiernan](https://discuss.hail.is/u/Elizabeth_Kiernan)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:58pm UTC](https://discuss.hail.is/t/error-running-hail-0-2-134-py3-11/4201 "2025-10-29T19:58:50Z")
Hi Hail team, I’m converting a large VDS to VCF via WDL and hitting what looks like a partioning-related error: Traceback (most recent call last): File “”, line 69, in File “”, line 2, in export\_vcf File “/usr/l… --- ## [Snp\_matrix \<- fread("File.raw") Out of Memory](https://discuss.hail.is/t/snp-matrix-fread-file-raw-out-of-memory/4202)
**Author:** [@HugHo](https://discuss.hail.is/u/HugHo)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:54pm UTC](https://discuss.hail.is/t/snp-matrix-fread-file-raw-out-of-memory/4202 "2025-10-29T19:54:54Z")
Hello, I would like to perform a GWAS and I already have the .map and raw. files with the phenotype file but the raw file weight 4Go ( output ls -l on puhti server -rw-rw---- 1 huguetho huguetho 4004201413 Aug 4 13:10 … --- ## [Hail not working with recent versions of VEP](https://discuss.hail.is/t/hail-not-working-with-recent-versions-of-vep/4204)
**Author:** [@sk4](https://discuss.hail.is/u/sk4)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:41pm UTC](https://discuss.hail.is/t/hail-not-working-with-recent-versions-of-vep/4204 "2025-10-29T19:41:42Z")
Hi! I have Hail and VEP installed as modules on an HPC cluster, and I am trying to perform functional annotation on an imported VCF using VEP through Hail in order to distribute the task across multiple nodes. However, … --- ## [HailException: array index out of bounds when converting Hail Table to Pandas DataFrame](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193)
**Author:** [@hrithikgupta88](https://discuss.hail.is/u/hrithikgupta88)\ **Replies:** 2\ **Last updated:** [October 29, 2025, 7:32pm UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193 "2025-10-29T19:32:53Z")
Description: We are processing VCF files stored in S3 using Hail in Python, and encountering the following error when calling .to\_pandas() on a selected Hail Table: hail.utils.java.HailException: array index out of bou… --- ## [Hail index file compression type](https://discuss.hail.is/t/hail-index-file-compression-type/4196)
**Author:** [@PHILIPP111007](https://discuss.hail.is/u/PHILIPP111007)\ **Replies:** 4\ **Last updated:** [October 29, 2025, 7:27pm UTC](https://discuss.hail.is/t/hail-index-file-compression-type/4196 "2025-10-29T19:27:58Z")
Hello, what type of compression does the index file have? (matrix\_table/index/part-0/index) Sincerely, Roshchin Philipp --- ## [Does liftover update alleles](https://discuss.hail.is/t/does-liftover-update-alleles/4205)
**Author:** [@hyslin](https://discuss.hail.is/u/hyslin)\ **Replies:** 1\ **Last updated:** [October 29, 2025, 7:08pm UTC](https://discuss.hail.is/t/does-liftover-update-alleles/4205 "2025-10-29T19:08:16Z")
When I applied liftover from GRCh38 to GRCh37 on my Hail table, the process was pretty fast but it didn’t seem to update the alleles. Does liftover also change the alleles? If not, what is the recommended way to update t… --- ## [Merging new genotype fields](https://discuss.hail.is/t/merging-new-genotype-fields/4208)
**Author:** [@handecadir](https://discuss.hail.is/u/handecadir)\ **Replies:** 0\ **Last updated:** [October 23, 2025, 3:02pm UTC](https://discuss.hail.is/t/merging-new-genotype-fields/4208 "2025-10-23T15:02:07Z")
Dear Hail community, We have a matrix table we have created from 92 cases and 100 controls. These were not joint variant called together. Cases were joint called and controls were also joint called seperately. We have … --- ## [Convert Clair3 gVCF to VDS failed with NumberFormatException error](https://discuss.hail.is/t/convert-clair3-gvcf-to-vds-failed-with-numberformatexception-error/4207)
**Author:** [@Sol\_Li](https://discuss.hail.is/u/Sol_Li)\ **Replies:** 3\ **Last updated:** [October 7, 2025, 2:07pm UTC](https://discuss.hail.is/t/convert-clair3-gvcf-to-vds-failed-with-numberformatexception-error/4207 "2025-10-07T14:07:04Z")
Hi, Hope to get some insight into the issue I had above and thanks in advanced. I was trying to import 2 gVCF files named “sample1.wf\_snp.gvcf.gz” and “sample2.wf\_snp.gvcf.gz” intp hail and converting them into VDS wit… --- ## [Summarize\_variants divide by zero error](https://discuss.hail.is/t/summarize-variants-divide-by-zero-error/4200)
**Author:** [@jjfarrell](https://discuss.hail.is/u/jjfarrell)\ **Replies:** 1\ **Last updated:** [September 4, 2025, 1:15pm UTC](https://discuss.hail.is/t/summarize-variants-divide-by-zero-error/4200 "2025-09-04T13:15:12Z")
Using version 0.2.132-678e1f52b999…. sum\_all=hl.summarize\_variants(mt,show=False) This line is generating an error when dividing by 0 when calculating the Ti/Tv ratio. The matrix table contains only structural variant… [Next page](https://discuss.hail.is/latest.md?page=1)