# Filtering out samples using hl.is\_nan

**URL:** <https://discuss.hail.is/t/filtering-out-samples-using-hl-is-nan/889>\
**Category:** Hail Query & hailctl\
**Created:** [March 1, 2019, 7:36pm UTC](https://discuss.hail.is/t/filtering-out-samples-using-hl-is-nan/889 "2019-03-01T19:36:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![hhx037](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/hhx037/32/160_2.png) [@hhx037](https://discuss.hail.is/u/hhx037)\
**Post date:** [March 1, 2019, 7:36pm UTC](https://discuss.hail.is/t/filtering-out-samples-using-hl-is-nan/889/1 "2019-03-01T19:36:35Z")

</div>

I want to remove samples with missing phenotypes from the matrix, and to do so I use `mt = mt.filter_cols(~hl.is_nan(mt.pheno))`  
It appears to indeed exclude samples, but if I test the opposite, keeping the samples with a missing phenotype, `mt = mt.filter_cols(hl.is_nan(mt.pheno))`, I get 0 when I count the columns, so I’m worried it’s not working the way I think it is and that the exclusion I see does not truly correspond to what I intend to do.

Could anyone explain it to me please?

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [March 2, 2019, 6:44pm UTC](https://discuss.hail.is/t/filtering-out-samples-using-hl-is-nan/889/2 "2019-03-02T18:44:31Z")

</div>

nan and missing are totally different. `mt = mt.filter_cols(~hl.is_nan(mt.pheno))` is actually doing what you want, though, by accident – when the filter condition evaluates to missing, the col is removed. Since `hl.is_nan(mt.pheno)` is missing when `mt.pheno` is missing, this removes missing phenotypes.

When you flip it and do `mt.filter_cols(hl.is_nan(mt.pheno)`, then the missing phenotypes will still be missing, and the defined phenotypes aren’t nan (presumably) and return false. So everything is filtered.

`hl.is_missing` is the right function to use here

---

<div class="post-metadata">

**Author:** ![hhx037](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/hhx037/32/160_2.png) [@hhx037](https://discuss.hail.is/u/hhx037)\
**Post date:** [March 3, 2019, 9:34am UTC](https://discuss.hail.is/t/filtering-out-samples-using-hl-is-nan/889/3 "2019-03-03T09:34:02Z")

</div>

Oh, I see, makes sense now! Thank you 🙂