# Linear\_regression with multiple phenotypes in Hail 0.2 **URL:** \ **Category:** Hail Query & hailctl\ **Created:** [August 23, 2018, 9:01pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621 "2018-08-23T21:01:06Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Wei\_Zhang](https://avatars.discourse-cdn.com/v4/letter/w/3d9bf3/32.png) [@Wei\_Zhang](https://discuss.hail.is/u/Wei_Zhang)\ **Post date:** [August 23, 2018, 9:01pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/1 "2018-08-23T21:01:06Z")
Hi, I have a problem when using linear\_regression with multiple phenotypes in Hail 0.2. Please see below. Basically I’m trying to run linear regressions with 501 phenotypes. Then I got an error message “Method code too large” ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/97825bb1a1a6acc98ccdd508364d5f78788c27be.png) ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/47cdb9a11eded01ca4aba2bddcfc2db6f1378c21.png) Any suggestions would be appreciated. Thank you! ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [August 23, 2018, 9:04pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/2 "2018-08-23T21:04:02Z")
A better alternative to `eval`: ```auto ys = [mt2_g_cis['y']] + [mt2_G_cis[f'p{i+1}'] for i in range(nPermutations)] ``` What’s the full error message here? The error is likely from something earlier in the pipeline. ---
**Author:** ![Wei\_Zhang](https://avatars.discourse-cdn.com/v4/letter/w/3d9bf3/32.png) [@Wei\_Zhang](https://discuss.hail.is/u/Wei_Zhang)\ **Post date:** [August 23, 2018, 9:19pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/3 "2018-08-23T21:19:15Z")
Thanks for the quick reply! Please see below for the full error message: Name: org.apache.toree.interpreter.broker.BrokerException Message: Traceback (most recent call last): File “/tmp/kernel-PySpark-c73f7ddf-473e-4623-8cda-b17160463cf5/pyspark\_runner.py”, line 194, in eval(compiled\_code) File “”, line 6, in File “/mnt/tmp/spark-37a03cc9-7ba1-4c06-bdde-fad17c5f603e/userFiles-b93fc739-9084-4b7a-93e5-95db1180b7b0/hail-python.zip/hail/typecheck/check.py”, line 547, in wrapper return f(\*args\_, \*\*kwargs\_) File “/mnt/tmp/spark-37a03cc9-7ba1-4c06-bdde-fad17c5f603e/userFiles-b93fc739-9084-4b7a-93e5-95db1180b7b0/hail-python.zip/hail/methods/statgen.py”, line 361, in linear\_regression block\_size) File “/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/java\_gateway.py”, line 1133, in **call** answer, self.gateway\_client, self.target\_id, self.name) File “/mnt/tmp/spark-37a03cc9-7ba1-4c06-bdde-fad17c5f603e/userFiles-b93fc739-9084-4b7a-93e5-95db1180b7b0/hail-python.zip/hail/utils/java.py”, line 196, in deco ‘Error summary: %s’ % (deepest, full, hail. **version** , deepest)) from None hail.utils.java.FatalError: RuntimeException: Method code too large! Java stack trace: java.lang.RuntimeException: Method code too large! at is.hail.relocated.org.objectweb.asm.MethodWriter.a(Unknown Source) at is.hail.relocated.org.objectweb.asm.ClassWriter.toByteArray(Unknown Source) at is.hail.asm4s.FunctionBuilder.classAsBytes(FunctionBuilder.scala:306) at is.hail.expr.ir.EmitFunctionBuilder.result(EmitFunctionBuilder.scala:284) at is.hail.expr.ir.Compile$.apply(Compile.scala:50) at is.hail.expr.ir.Compile$.apply(Compile.scala:31) at is.hail.expr.ir.CompileWithAggregators$$anonfun$4.apply(Compile.scala:170) at is.hail.expr.ir.CompileWithAggregators$$anonfun$4.apply(Compile.scala:170) at is.hail.expr.ir.MatrixMapCols.execute(MatrixIR.scala:1231) at is.hail.expr.ir.MatrixMapCols.execute(MatrixIR.scala:1143) at is.hail.expr.ir.MatrixMapEntries.execute(MatrixIR.scala:958) at is.hail.variant.MatrixTable.value$lzycompute(MatrixTable.scala:509) at is.hail.variant.MatrixTable.value(MatrixTable.scala:504) at is.hail.variant.MatrixTable.x$13$lzycompute(MatrixTable.scala:514) at is.hail.variant.MatrixTable.x$13(MatrixTable.scala:514) at is.hail.variant.MatrixTable.colValues$lzycompute(MatrixTable.scala:514) at is.hail.variant.MatrixTable.colValues(MatrixTable.scala:514) at is.hail.stats.RegressionUtils$.getColumnVariables(RegressionUtils.scala:64) at is.hail.stats.RegressionUtils$.getPhenosCovCompleteSamples(RegressionUtils.scala:93) at is.hail.methods.LinearRegression$.apply(LinearRegression.scala:26) at is.hail.methods.LinearRegression.apply(LinearRegression.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:748) Hail version: devel-ae9e34fb3cbf Error summary: RuntimeException: Method code too large! StackTrace: org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163) org.apache.toree.interpreter.broker.BrokerState$$anonfun$markFailure$1.apply(BrokerState.scala:163) scala.Option.foreach(Option.scala:257) org.apache.toree.interpreter.broker.BrokerState.markFailure(BrokerState.scala:162) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) py4j.Gateway.invoke(Gateway.java:280) py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) py4j.commands.CallCommand.execute(CallCommand.java:79) py4j.GatewayConnection.run(GatewayConnection.java:214) java.lang.Thread.run(Thread.java:748) ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [August 23, 2018, 9:19pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/4 "2018-08-23T21:19:46Z")
What’s the full python script? This is coming from an `annotate_cols` or `select_cols`. ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [August 23, 2018, 9:20pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/5 "2018-08-23T21:20:34Z")
I would also recommend strongly that you update to the latest version – yours is ~6 weeks old ---
**Author:** ![Wei\_Zhang](https://avatars.discourse-cdn.com/v4/letter/w/3d9bf3/32.png) [@Wei\_Zhang](https://discuss.hail.is/u/Wei_Zhang)\ **Post date:** [August 23, 2018, 9:21pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/6 "2018-08-23T21:21:57Z")
The code is as follows. Basically I annotated the mt by 500 permutations and run linear regressions. Please see below: # Annotate mt by real and permutated gene expression (residuals of 1 PEER factor, Age, HTN, and disease) table = (hl.import\_table(‘s3://gfb-genomics/gene2permsPilot\_G/{}’.format(gene), impute=True, missing=’’) .key\_by(‘Sample\_name’)) mt2\_G\_cis = mt2\_G\_cis.annotate\_cols(\*\*table[mt2\_G\_cis.s]) nPermutations = 500 # Run linreg\_multi\_pheno, ys = [eval(‘mt2\_G\_cis.y’)] + [eval(‘mt2\_G\_cis.p{}’.format(i+1)) for i in range(nPermutations)] # controlling for Gender and 4 Genotyping PCs mt2\_G\_cis\_eQTLs = hl.linear\_regression(y=ys, x=mt2\_G\_cis.GT.n\_alt\_alleles(), covariates=[1.0, mt2\_G\_cis.isFemale, mt2\_G\_cis.PC1, mt2\_G\_cis.PC2, mt2\_G\_cis.PC3, mt2\_G\_cis.PC4]) ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [August 23, 2018, 9:32pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/7 "2018-08-23T21:32:35Z")
what does that table look like? Does it have 20000 columns? To start, I would recommend doing something like ```auto mt2_G_cis = mt2_G_cis.annotate_cols(genes = table[mt2_G_cis.s]) ``` If you evaluate `dict(**table[mt2_G_cis.s])` you’ll see that this expands into something horrible. ---
**Author:** ![Wei\_Zhang](https://avatars.discourse-cdn.com/v4/letter/w/3d9bf3/32.png) [@Wei\_Zhang](https://discuss.hail.is/u/Wei_Zhang)\ **Post date:** [August 23, 2018, 10:07pm UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/8 "2018-08-23T22:07:34Z")
The table has 136 rows and 501 columns. So it is not huge at all. I’ll try your suggestions. Thank you! ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [August 24, 2018, 2:17am UTC](https://discuss.hail.is/t/linear-regression-with-multiple-phenotypes-in-hail-0-2/621/9 "2018-08-24T02:17:42Z")
If you try with the latest version and still get the error, please send us the `hail.log` file so we can more effectively debug. This will contain the computational graph that failed to execute