Convert Table field type

Seems simple enough but I am not figuring this out.

I have a table of sample annotations I am uploading with several columns coded as 0/1. I import those as tints (types={‘pheno’:hl.tint32).

I want to convert this to a bool so I can remove all filter_cols() to only keep the columns with this annotation. I would want 1 (or any non-zero annotation) to code to True, and 0 to code to False.

Setting types={‘pheno’:hl.tbool) from the getgo via import_table() doesn’t seem to work. I get FatalError: IllegalArgumentException: For input string: “0”.

How can I cast this int to a bool? I tried using eval() and transmute(). I know I can just change the input file with a variety of command line tools but I imagine this will be simple to do in Hail too

hl.eval will basically never appeal in pipelines – I realize our docs for functions are confusing in this sense.

I’d use transmute/annotate with something like:

ht = hl.import_table(..., types={'pheno': 'int'})
ht = ht.transmute(pheno = pheno == 1) # 1 => True, anything else => False
1 Like

Awesome, works great!

had to use:

ht = ht.transmute(pheno = ht.pheno == 1)

but did exactly what I want. Thanks!

yes, oops, typo!

How would I filter out columns where pheno is not None?

mt = mt.filter_cols(mt.pheno,keep=True) works to keep all the True instances.

mt.aggregate_cols(hl.agg.counter(mt.pheno)) gives me {False: 34820, True: 2878, None: 449711}

mt.aggregate_cols(hl.agg.count_where(mt.pheno == True)) tells me how many are true, but if I change to True to None I get “ExpressionException: Hail cannot impute the type of ‘None’”

If i have a bool as a column attribute that I have merged into a MatrixTable from a Table, what is the default “None” type used and is there an equivalent?

The None you are seeing is what we return when there is missing data. You can check for this with hl.is_missing(mt.pheno). That expression will be True for all the missing values.

1 Like

Try mt = mt.filter_cols(hl.is_defined(mt.pheno), keep=True).

Also, see the core functions for working with missing values: is_missing, coalesce, null, or_missing and or_else:

1 Like

Awesome! You folks rule. Still figuring out how to navigate throught the documentation pages, but trust I did look around! Thanks for the link.

The cheatsheets help:

1 Like

cool, somehow missed that!