# Filtering MatrixTables where column values do not match

**URL:** <https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557>\
**Category:** Hail Query & hailctl\
**Created:** [July 29, 2020, 1:55pm UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557 "2020-07-29T13:55:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![spencer](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@spencer](https://discuss.hail.is/u/spencer)\
**Post date:** [July 29, 2020, 1:55pm UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557/1 "2020-07-29T13:55:49Z")

</div>

Hi all, I’m relatively new to Hail and am having some difficulty filtering a MatrixTable by rows. I’ve read in a vcf and done some basic QC, giving me a MatrixTable I’ve named mts. I have 3 samples, and I need to remove all rows where GT is not 0/0 for all samples. When I run mts.GT.show() I can clearly see the values for each sample.

 ![Screen Shot 2020-07-28 at 5.30.23 PM](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/7ced558a4f11f2bfeedb1a05a4381677d2608cee.png)

Could someone point me in the right direction for filtering these rows?

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [July 29, 2020, 1:59pm UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557/2 "2020-07-29T13:59:08Z")

</div>

The MatrixTable interface intentionally doesn’t let you query single entries by column value, but instead forces you to write your pipeline in terms of computations applied all entry values as **aggregations**.

Here are two ways to remove sites that are 0/0 at every sample

Using aggregators:

```python
mts = mts.filter_rows(hl.agg.all(mt.GT.is_hom_ref()), keep=False)

```

Using variant\_qc (which uses aggregators in its implementation):

```auto
mts = hl.variant_qc(mts)
mts = mts.filter_rows(mts.variant_qc.AC[0] > 0)

```

---

<div class="post-metadata">

**Author:** ![spencer](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@spencer](https://discuss.hail.is/u/spencer)\
**Post date:** [July 29, 2020, 2:06pm UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557/3 "2020-07-29T14:06:36Z")

</div>

Is there a way to do the opposite, so remove all sites that are NOT 0/0? I’ve tried using `is_het_ref()` and `mt.GT.is_hom_ref() == False`, but both seem to give me the same output as `mts = mts.filter_rows(hl.agg.all(mt.GT.is_hom_ref()), keep=False)`

Update:  
`f = mts.filter_rows(mts.variant_qc.AC[0] == 6)` seems to work

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [July 30, 2020, 1:02am UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557/4 "2020-07-30T01:02:27Z")

</div>

> [@spencer](#):
>
> Update:  
> `f = mts.filter_rows(mts.variant_qc.AC[0] == 6)` seems to work

This is pretty tailored to your example. I think I might do something like:

```python
mts = mts.filter_rows(hl.agg.all(mt.GT.is_non_ref()), keep=False)

```

is\_non\_ref returns True if the call has at least one non-reference allele.

---

<div class="post-metadata">

**Author:** ![johnnyr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/johnnyr/32/572_2.png) [@johnnyr](https://discuss.hail.is/u/johnnyr)\
**Post date:** [February 22, 2021, 11:13am UTC](https://discuss.hail.is/t/filtering-matrixtables-where-column-values-do-not-match/1557/5 "2021-02-22T11:13:23Z")

</div>

I have tried to translate this to filter\_cols to remove all genotypes which do not show a variant by using this:

mtf2 = mtf.filter\_cols(hl.agg.all(mtf.GT.is\_het\_non\_ref()), keep=False)

which in principle seems to work, as mtf.count() gives (4, 1680) and mtf2.count() (4, 517). However, I have still genotypes left which do have a combination of 0/0 and NA. So I assume having NA as genotype is a problem. Is there a way to also remove those ?
