String to complex expression

Hail Query & hailctl
1

If I have a complex hail expression stored in a string, for example:

exprStr = “hl.min(hl.agg.call_stats(mt.GT, mt.alleles).AF)”

Can I execute this with hail? for example

mt = mt.annotate_rows(MAF = CONVERT_2_EXPR(exprStr))

I want to be able to pass any sort of complex expression to annotate_rows through exprStr

Thanks

2

It is possible to convert the whole statement into a string like below
and then used the python exec built-in function to execute it.
I test it and it works.
Is there any complication in your opinion?
yet I am interested to do it in a way that only the expression is in a string.

statement = "mt = mt.annotate_rows(MAF = hl.min(hl.agg.call_stats(mt.GT, mt.alleles).AF))"
exec(statement)
3

There’s no reason this shouldn’t work, but exec / eval are pretty dangerous. See here (and search around elsewhere if interested too):

I suppose beyond the general issues that exec/eval present, the Hail-specific complication is that your string relies on the name mt and won’t work (or may do something different) if the matrix table is named something else.

4

Can you share more of your motivation? It sounds like you want to programmatically generate Hail expressions, You certainly can do this! For example, you might write a function which finds the minimum af:

def minor_af(mt: hl.MatrixTable):
    return hl.min(hl.agg.call_stats(mt.GT, mt.alleles).AF)

and use it:

mt = mt.annotate_rows(MAF = minor_af(mt))
5

Thanks for your reply. You are right, I’m a bit worry about exec() and I am aware of the use of mt in the string but it is ok in my case.
I think of writing a function to parse the string and somehow form the expression but it would be quiet a lot of work.

6

Thanks for your reply. Yes I agree it is a bit weird why I am trying to do so.

I am working on a project called Cohort Analysis Platform (CAP) in Garvan Institute of Medical Research.
The project is all about Codeless Analysis at Scale where user describe analysis in a YAML file and then CAP take care of it.
CAP is going to utilise many tools (i.e. VEP, Plink, PRSice) but it mainly use Hail for almost everything.

I want to be able to have Hail Expression in the YAML file (i.e. for annotation or aggregation) and then CAP execute it. That is way I need to convert a string to a expression

I want to have ultimate flexibility in my YAML description and not limited in a few expression implemented internal in CAP

The tutorial page on the GitHub may give you an idea how CAP is going to work: CAP/TUTORIAL.md at main · ArashLab/CAP · GitHub

But the code has been changed significantly since then and is subject to more changes.
Also it is pretty a mess and incomplete at the moment until I make v0.2.0

7

Ah, I understand. If you want folks to write valid Python code in your YAML, then I think eval is the right thing to do. You’ll need to pick name for the matrix table in question or have the users specify the name they’re using to refer to the mt. Then you can specify the matrix table with that name in the globals or locals argument to eval.

8

Thanks for the comment. It was helpful.