Merge vcfs like bcftools merge

[njbernstein]

Hi there,

I’m wondering if there is a best practice for merging VCFs together in hail, akin to the bcftools merge function.

Best
Nick

[tpoterba]

What kind of merge? Inner/outer on variants, samples, etc?

[njbernstein]

Sorry about not having enough info.

I have 50k sample vcfs, and I want to merge them into single vcf keeping all variants found in any sample (outer).

[tpoterba]

Are these GVCFs or VCFs? If the former, there’s a good solution. The latter, it’s going to be very difficult/impossible to get a sensible result, partly because VCFs are lossy (no reference blocks)

[njbernstein]

It’s the latter. I’m okay with a bunch of no calls in the merged VCF with all the samples because these are somatic variant calls, and I don’t expect any variant to be common. If that was your concern.

Or is there another concern I’m not catching?

[tpoterba]

this is a duplicate of:

Outer Joining Multiple VCFs

Hello! I wanted to see if there’s a simple way to outer join multiple VCFs in Hail. I have ~50,000 VCFs each of which has somatic calls from 1 individual, and basically my goal is to perform QC on each VCF (ex: removing non-PASS variants and low-qual genotypes), then join all the 50,000 VCFs into 1 hail .mt (I don’t need any entry-level info on each variant except for GT and VAF, VEP-annotate that .mt, and export_entries on for downstream analysis. Would be fantastic if there was a way to do t…

This is possible to do by writing a hierarchical merge with intermediate checkpoints, but it’ll probably take an hour or two to write down the correct code for that and I don’t have time right now.

[njbernstein]

Oh thats funny. Alright. I’ll try writing something up. Thanks!