# Slow speed when using gnomadV3 callset

**URL:** <https://discuss.hail.is/t/slow-speed-when-using-gnomadv3-callset/3827>\
**Category:** Hail Query & hailctl\
**Created:** [May 8, 2024, 6:47am UTC](https://discuss.hail.is/t/slow-speed-when-using-gnomadv3-callset/3827 "2024-05-08T06:47:40Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![DBScan](https://avatars.discourse-cdn.com/v4/letter/d/bc79bd/32.png) [@DBScan](https://discuss.hail.is/u/DBScan)\
**Post date:** [May 8, 2024, 6:47am UTC](https://discuss.hail.is/t/slow-speed-when-using-gnomadv3-callset/3827/1 "2024-05-08T06:47:40Z")

</div>

I am trying to build my own ancestry classifier using genotypes from the 1000Genomes and HGDP dataset, taken from here [Downloads | gnomAD (broadinstitute.org)](https://gnomad.broadinstitute.org/downloads#v3-hgdp-1kg). The speed I observe is very slow (for example during LD pruning and filtering), compared to when I analyzed my own dataset. In both approaches, I initialize HAIL like this:

```python
hl.init(tmp_dir = '/scratch/hail',
        local_tmpdir = '/scratch/hail',
        master='local[128]',
        spark_conf={'spark.driver.memory': '1800g',
                    'spark.executor.memory': '1800g',
                    'spark.local.dir' : '/scratch/hail',
                   'java.io.tmpdir': '/scratch/hail'}) 

```

I’ve noticed that the gnomad dataset contains 50’000 partitions, so I tried to change them during `read_matrix_table`, but I didnt observe a difference. Is there anything else I could try?
