Is there a way in Hail, within filtering operations, to combine positive and negative logical statements?
e.g. (matrix.X.contains(a_value)) !& (matrix.Y.contains(another_value))
An example here is filtering out clinvar benign variants from a data set following VEP annotation. I couldn’t find filtering conditions that also allowed a row to be kept if the clinvar annotations were entirely missing values. The logic I would like to use looks like:
matrix = matrix.filter_rows((not hl.is_missing(clinvar)) & ((clinvar.contains(benign) & (clinvar.stars > 0)), keep=False)
As there is missing data present in the clinvar annotations, I had to use a replacement step to make sure the logical operations on the values were accurately carried out
clinvar = hl.if_else(clinvar, hl.missing(hl.str())
I’ve tried using a few different variations:
- ‘not’ operator
- !(logical condition)
- !& operator (I’m not sure if this exists in python at all…)
I think I’ve checked through all the documentation and didn’t find anything that was a good fit. I don’t mind using the replacement method, but I have quite a few similar conditions in mind, and doubling the number of operations instead of offering an alternative path in the logic is reasonably expensive.
Please let me know if something like this exists in the current syntax!