Convert Table field type

Hail Query & hailctl
1

Seems simple enough but I am not figuring this out.

I have a table of sample annotations I am uploading with several columns coded as 0/1. I import those as tints (types={‘pheno’:hl.tint32).

I want to convert this to a bool so I can remove all filter_cols() to only keep the columns with this annotation. I would want 1 (or any non-zero annotation) to code to True, and 0 to code to False.

Setting types={‘pheno’:hl.tbool) from the getgo via import_table() doesn’t seem to work. I get FatalError: IllegalArgumentException: For input string: “0”.

How can I cast this int to a bool? I tried using eval() and transmute(). I know I can just change the input file with a variety of command line tools but I imagine this will be simple to do in Hail too

2

hl.eval will basically never appeal in pipelines – I realize our docs for functions are confusing in this sense.

I’d use transmute/annotate with something like:

ht = hl.import_table(..., types={'pheno': 'int'})
ht = ht.transmute(pheno = pheno == 1) # 1 => True, anything else => False
1 Like
3

Awesome, works great!

had to use:

ht = ht.transmute(pheno = ht.pheno == 1)

but did exactly what I want. Thanks!

4

yes, oops, typo!

5

How would I filter out columns where pheno is not None?

mt = mt.filter_cols(mt.pheno,keep=True) works to keep all the True instances.

mt.aggregate_cols(hl.agg.counter(mt.pheno)) gives me {False: 34820, True: 2878, None: 449711}

mt.aggregate_cols(hl.agg.count_where(mt.pheno == True)) tells me how many are true, but if I change to True to None I get “ExpressionException: Hail cannot impute the type of ‘None’”

If i have a bool as a column attribute that I have merged into a MatrixTable from a Table, what is the default “None” type used and is there an is.na() equivalent?

6

The None you are seeing is what we return when there is missing data. You can check for this with hl.is_missing(mt.pheno). That expression will be True for all the missing values.

1 Like
7

Try mt = mt.filter_cols(hl.is_defined(mt.pheno), keep=True).

Also, see the core functions for working with missing values: is_missing, coalesce, null, or_missing and or_else: https://hail.is/docs/0.2/functions/core.html.

1 Like
8

Awesome! You folks rule. Still figuring out how to navigate throught the documentation pages, but trust I did look around! Thanks for the link.

9

The cheatsheets help: https://hail.is/docs/0.2/cheatsheets.html

1 Like
10

cool, somehow missed that!