# Remove rows with "NA" in any sample

**URL:** <https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374>\
**Category:** Hail Query & hailctl\
**Created:** [November 27, 2021, 4:57pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374 "2021-11-27T16:57:33Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [November 27, 2021, 4:57pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/1 "2021-11-27T16:57:33Z")

</div>

From a matrix table, how do I remove rows (variants) with a “NA” in any one of the samples? Thank you!

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [November 29, 2021, 3:06pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/2 "2021-11-29T15:06:41Z")

</div>

Hi @CuriousGeneticist !

> From a matrix table, how do I remove rows with an NA in any one of the samples?

We can take your sentence piece-by-piece from English to Hail!

> From a matrix table,

This tells us that we’ll be using [MatrixTable methods](https://hail.is/docs/0.2/hail.MatrixTable.html#hail.MatrixTable) like `annotate_rows`, `filter_rows`, `annotate_cols`, etc. instead of the Table methods `annotate` and `filter`.

> remove rows (variants) with

Removal of rows or columns in Hail is called “filtering”, so we want `filter_rows`:

```python
mt = mt.filter_rows( some_condition )

```

> a “NA” in any one of the samples

This has three parts “a ‘NA’”, “any one of”, and “the samples” which correspond to:

1. `hl.is_missing( some_thing_that_can_be_missing )`
2. `hl.agg.any( some_other_condition )`
3. `mt.GT`, `mt.AD`, … (the various genotype fields which vary per-sample)

I will assume you want to look for missing genotypes (`mt.GT`).

```python
mt = mt.filter_rows(
    hl.agg.any(
        hl.is_missing(mt.GT)
    )
)

```

* * *

An important caveat! Hail has both “missing” entry fields and “filtered entries”. Filtered entries are created by `filter_entries`. You never get filtered entries from `import_vcf`. You can check if you have any filtered entries by running this command:

```python
mt = mt.compute_entry_filter_stats()
mt.entry_stats_row.n_filtered.summarize()

```

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 2, 2021, 6:38pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/3 "2021-12-02T18:38:19Z")

</div>

> [@danking](#):
>
> ```python
> mt = mt.filter_rows(
> hl.agg.any(
> hl.is_missing(mt.GT)
> )
> )
> 
> ```

Thank you so much for your response Danking. I tried to run this code:  
_mt = mt.filter\_rows(_

- hl.agg.any(\*
- 

```
   hl.is_missing(mt.GT)*

```

- )\*  
_)_  
But the NA rows are still there. Does that mean that it is a “filtered entry”?

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 2, 2021, 6:41pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/4 "2021-12-02T18:41:37Z")

</div>

> But the NA rows are still there

What is the query you are running to see the “NA rows”? This might help us best answer.

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 3, 2021, 5:40pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/5 "2021-12-03T17:40:26Z")

</div>

I used  
_mt.GT.show()_  
and I still saw the “NA” rows.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [December 3, 2021, 5:43pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/6 "2021-12-03T17:43:21Z")

</div>

Can you copy and paste or screenshot an example row from `mt.GT.show()`? Also what is printed by my suggestion above for checking for filtered entries?

> [@danking](#):
>
> You can check if you have any filtered entries by running this command:
> 
> ```python
> mt = mt.compute_entry_filter_stats()
> mt.entry_stats_row.n_filtered.summarize()
> 
> ```

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 3, 2021, 6:05pm UTC](https://discuss.hail.is/t/remove-rows-with-na-in-any-sample/2374/7 "2021-12-03T18:05:19Z")

</div>

Actually, I realized that it was an issue with an upstream query. Thanks so much for your help!