# Fail to read a T2T VCF

**URL:** <https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989>\
**Category:** Feature Requests\
**Created:** [December 13, 2022, 11:41am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989 "2022-12-13T11:41:28Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![VinceCarey](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@VinceCarey](https://discuss.hail.is/u/VinceCarey)\
**Post date:** [December 13, 2022, 11:41am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/1 "2022-12-13T11:41:29Z")

</div>

Trying to import one of the VCFs found [here](https://anvil.terra.bio/#workspaces/anvil-datastorage/AnVIL_T2T)

The risk was high – but the closest reference available is GRCh38

```python
h.init(default_reference='GRCh38')
h.import_vcf("chr17.genotyped.vcf.gz", force_bgz=True).write("t2t17.mt")

```

```python
Error summary: HailException: Invalid locus 'chr17:83402572' found. Position '83402572' is not within the range [1-83257441] for reference genome 'GRCh38'.
[Stage 0:>   

```

There’s probably a workaround defining the allowable ranges? But I could not find it and I think the doc on reference genome identification and modification could be enhanced to directly address the use case: I need to specify a complete new reference. Or I missed it. I see how to create a ReferenceGenome instance from JSON but I don’t see how to impose it in the hail instance.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 13, 2022, 11:44am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/2 "2022-12-13T11:44:17Z")

</div>

this should work:

```python
rg = hl.ReferenceGenome.read(json_file)
hl.import_vcf("chr17.genotyped.vcf.gz", force_bgz=True, reference_genome=rg)

```

I don’t think you can set the default RG to a custom one – this requires a slight redesign of the client session model and isn’t super high priority right now.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 13, 2022, 11:45am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/3 "2022-12-13T11:45:39Z")

</div>

Should only need to do this once, though, since reading a table/matrixtable with a custom reference will load that reference into the session. In the future you can read that MT and do `rg = mt.locus.dtype.reference_genome` if you need to use it elsewhere.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 13, 2022, 11:45am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/4 "2022-12-13T11:45:40Z")

</div>

Should only need to do this once, though, since reading a table/matrixtable with a custom reference will load that reference into the session. In the future you can read that MT and do `rg = mt.locus.dtype.reference_genome` if you need to use it elsewhere.

---

<div class="post-metadata">

**Author:** ![VinceCarey](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@VinceCarey](https://discuss.hail.is/u/VinceCarey)\
**Post date:** [December 13, 2022, 11:48am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/5 "2022-12-13T11:48:19Z")

</div>

Thanks so much for such rapid response! Before I go after this, is it sufficient to define the reference genome in json for the single contig of interest, or do I need to define the whole genome?

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 13, 2022, 11:50am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/6 "2022-12-13T11:50:35Z")

</div>

Defining a RG with a single contig will work if your VCF only has data on that chromosome, but if you later need to work with multiple contigs, you’ll need to either reimport or define a liftover mapping from your chr17-only-rg to a more complete reference genome.

---

<div class="post-metadata">

**Author:** ![VinceCarey](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@VinceCarey](https://discuss.hail.is/u/VinceCarey)\
**Post date:** [December 13, 2022, 2:12pm UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/7 "2022-12-13T14:12:43Z")

</div>

Success! It required some conniving though, because the VCF had more positions than NCBI claims for the contig. When I get a proper reference genome spec I will post it here for posterity.

---

<div class="post-metadata">

**Author:** ![VinceCarey](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@VinceCarey](https://discuss.hail.is/u/VinceCarey)\
**Post date:** [January 7, 2023, 12:01am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/8 "2023-01-07T00:01:04Z")

</div>

To bring this to closure [this json file](https://raw.githubusercontent.com/vjcitn/BiocHail/main/inst/json/t2tAnVIL.json) gives chromosome lengths for autosomes of the T2T reference, and can be used with ReferenceGenome.read to work with VCFs created with this reference.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [January 7, 2023, 1:11am UTC](https://discuss.hail.is/t/fail-to-read-a-t2t-vcf/2989/9 "2023-01-07T01:11:04Z")

</div>

Thanks for sharing!
