# HailException: array index out of bounds when converting Hail Table to Pandas DataFrame

**URL:** <https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193>\
**Category:** Hail Query & hailctl\
**Created:** [August 12, 2025, 2:12pm UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193 "2025-08-12T14:12:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![hrithikgupta88](https://avatars.discourse-cdn.com/v4/letter/h/ac91a4/32.png) [@hrithikgupta88](https://discuss.hail.is/u/hrithikgupta88)\
**Post date:** [August 12, 2025, 2:12pm UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193/1 "2025-08-12T14:12:36Z")

</div>

**Description:**  
We are processing VCF files stored in S3 using Hail in Python, and encountering the following error when calling .to\_pandas() on a selected Hail Table:

hail.utils.java.HailException: array index out of bounds: index=3, length=3

**Relevant Code Context:**

- We download the VCF from S3 to a local temp path.

- We load it using:

mt = hl.import\_vcf(temp\_file\_path, force\_bgz=True, reference\_genome=reference\_clean,skip\_invalid\_loci=True, min\_partitions=300, array\_elements\_required=False)

- We split multi-allelic sites:

bi = mt.filter\_rows(hl.len(mt.alleles) == 2).annotate\_rows(a\_index=1, was\_split=False)

multi = mt.filter\_rows(hl.len(mt.alleles) \> 2)

split = hl.split\_multi\_hts(multi)

mt = split.union\_rows(bi)

- We annotate with RSID and extract the entries table:

row\_key\_table = mt.rows().key\_by(‘locus’, ‘alleles’).select(‘rsid’)

entries\_table = mt.entries()

entries\_table = entries\_table.annotate(ID=row\_key\_table[entries\_table.locus, entries\_table.alleles].rsid)

- Then we select columns:

selected\_table = entries\_table.select(

CHROM=entries\_table.locus.contig,

POS=entries\_table.locus.position,

ID=entries\_table.ID,

REF=entries\_table.alleles[0],

ALT=entries\_table.alleles[1:], # list to preserve multi-ALT

QUAL=entries\_table.qual,

FILTER=hl.if\_else(hl.len(entries\_table.filters) == 0, hl.array([‘PASS’]), hl.array(entries\_table.filters)),

ORIGINAL\_LOCUS=entries\_table.original\_locus,

ORIGINAL\_ALLELES=entries\_table.original\_alleles,

\*\*{f’FORMAT.{f}': entries\_table[f] for f in format\_fields\_filtered if f in entries\_table.row},

\*\*{f’INFO.{k}': entries\_table.info[k] for k in entries\_table.info}

)

- The error occurs when calling:

df = selected\_table.to\_pandas()

**Additional Notes:**

- Error happens only on certain VCFs, not all.

- The VCFs are annotated from different tools (snpEff, VEP, BCF).

- We have confirmed the file has valid variant rows before loading.

- Hail version: 0.2.78

- Full error log is attached for reference.

**Questions:**

1. What could cause this array index out of bounds in to\_pandas() for entries\_table/selected\_table?

2. Could it be related to ALT=entries\_table.alleles[1:] handling multi-allelic records or missing values after split?

3. Any best practices to defensively handle such cases before converting to Pandas?

---

<div class="post-metadata">

**Author:** ![hrithikgupta88](https://avatars.discourse-cdn.com/v4/letter/h/ac91a4/32.png) [@hrithikgupta88](https://discuss.hail.is/u/hrithikgupta88)\
**Post date:** [August 18, 2025, 7:15am UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193/2 "2025-08-18T07:15:52Z")

</div>

Hi @tpoterba

Could you please share any insights on this issue, in case you’ve come across something similar before?

---

<div class="post-metadata">

**Author:** ![patrick-schultz](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/patrick-schultz/32/265_2.png) [@patrick-schultz](https://discuss.hail.is/u/patrick-schultz)\
**Post date:** [October 29, 2025, 7:32pm UTC](https://discuss.hail.is/t/hailexception-array-index-out-of-bounds-when-converting-hail-table-to-pandas-dataframe/4193/3 "2025-10-29T19:32:53Z")

</div>

Hi @hrithikgupta88,

Sorry for the late response. Is this still a blocking issue for you? It is definitely a bug, but it would be hard for us to debug without access to a reproducing example.