# Merge vcfs like bcftools merge

**URL:** <https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361>\
**Category:** Hail Query & hailctl\
**Created:** [April 3, 2020, 5:37pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361 "2020-04-03T17:37:57Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![njbernstein](https://avatars.discourse-cdn.com/v4/letter/n/e9bcb4/32.png) [@njbernstein](https://discuss.hail.is/u/njbernstein)\
**Post date:** [April 3, 2020, 5:37pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/1 "2020-04-03T17:37:57Z")

</div>

Hi there,

I’m wondering if there is a best practice for merging VCFs together in hail, akin to the `bcftools merge` function.

Best  
Nick

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [April 3, 2020, 6:12pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/2 "2020-04-03T18:12:34Z")

</div>

What kind of merge? Inner/outer on variants, samples, etc?

---

<div class="post-metadata">

**Author:** ![njbernstein](https://avatars.discourse-cdn.com/v4/letter/n/e9bcb4/32.png) [@njbernstein](https://discuss.hail.is/u/njbernstein)\
**Post date:** [April 3, 2020, 7:45pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/3 "2020-04-03T19:45:23Z")

</div>

Sorry about not having enough info.

I have 50k sample vcfs, and I want to merge them into single vcf keeping all variants found in any sample (outer).

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [April 3, 2020, 7:55pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/4 "2020-04-03T19:55:26Z")

</div>

Are these GVCFs or VCFs? If the former, there’s a good solution. The latter, it’s going to be very difficult/impossible to get a sensible result, partly because VCFs are lossy (no reference blocks)

---

<div class="post-metadata">

**Author:** ![njbernstein](https://avatars.discourse-cdn.com/v4/letter/n/e9bcb4/32.png) [@njbernstein](https://discuss.hail.is/u/njbernstein)\
**Post date:** [April 3, 2020, 8:04pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/5 "2020-04-03T20:04:39Z")

</div>

It’s the latter. I’m okay with a bunch of no calls in the merged VCF with all the samples because these are somatic variant calls, and I don’t expect any variant to be common. If that was your concern.

Or is there another concern I’m not catching?

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [April 3, 2020, 8:14pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/6 "2020-04-03T20:14:51Z")

</div>

this is a duplicate of:

> [@Outer Joining Multiple VCFs](https://discuss.hail.is/t/outer-joining-multiple-vcfs/1350):
>
> Hello! I wanted to see if there’s a simple way to outer join multiple VCFs in Hail. I have ~50,000 VCFs each of which has somatic calls from 1 individual, and basically my goal is to perform QC on each VCF (ex: removing non-PASS variants and low-qual genotypes), then join all the 50,000 VCFs into 1 hail .mt (I don’t need any entry-level info on each variant except for GT and VAF, VEP-annotate that .mt, and export\_entries on for downstream analysis. Would be fantastic if there was a way to do t…

This is possible to do by writing a hierarchical merge with intermediate checkpoints, but it’ll probably take an hour or two to write down the correct code for that and I don’t have time right now.

---

<div class="post-metadata">

**Author:** ![njbernstein](https://avatars.discourse-cdn.com/v4/letter/n/e9bcb4/32.png) [@njbernstein](https://discuss.hail.is/u/njbernstein)\
**Post date:** [April 3, 2020, 8:32pm UTC](https://discuss.hail.is/t/merge-vcfs-like-bcftools-merge/1361/7 "2020-04-03T20:32:16Z")

</div>

Oh thats funny. Alright. I’ll try writing something up. Thanks!
