Merging overlapping & non-overlapping variants & variant annotations (no GT-level info needed) in hail, similar to R's merge() function

Feature Requests
1

Looking for something similar to R’s merge() function which allows for merging of overlapping and non-overlapping variants and annotations in Hail. I need this for the purpose of meta-analyzing single variant results from 3 different vdses, and annotating the resulting output.

2

format envisioned:

hail  importannotations table ${VDS1.bgz} \
  -c 'va.rsID = table.rsID, va.Gene = table.Gene, va.Consq = table.Consq,
  va.TOPMed.AF = table.AF, va.TOPMed.pval = table.pval, va.TOPMed.beta = table.beta, va.TOPMed.se = table.se, 
  va.TOPMed.nHomRef = table.nHomRef,  va.TOPMed.nHet = table.nHet,  va.TOPMed.nHomVar = table.nHomVar)' -e v --impute \
  importannotations table ${VDS2.bgz} \
  -c 'va.rsID = table.rsID, va.Gene = table.Gene, va.Consq = table.Consq,
  va.MESA.AF = table.AF, va.MESA.pval = table.pval, va.MESA.beta = table.beta, va.MESA.se = table.se, 
  va.MESA.nHomRef = table.nHomRef,  va.MESA.nHet = table.nHet,  va.MESA.nHomVar = table.nHomVar)' -e v --impute \
  importannotations table ${VDS3.bgz} \
  -c 'va.rsID = table.rsID, va.Gene = table.Gene, va.Consq = table.Consq,
  va.MESA.AF = table.AF, va.MESA.pval = table.pval, va.MESA.beta = table.beta, va.MESA.se = table.se, 
  va.MESA.nHomRef = table.nHomRef,  va.MESA.nHet = table.nHet,  va.MESA.nHomVar = table.nHomVar)' -e v --impute \ ...[doing meta-analysis and exportingvariants]
3

Hi Maryam, this should already be possible using KeyTables and outer join in the Python front end.

https://hail.is/pyhail/index.html#pyhail.KeyTable

Let’s work this out together tomorrow and we can then post the workflow here.

4

Hi,

have you worked out a way to do this? I’m also interested in the possibility of doing meta-analyses of several vds sets in Hail, is that possible?