# py4j.protocol.Py4JNetworkError: Answer from Java side is empty **URL:** \ **Category:** Hail Query & hailctl\ **Created:** [April 29, 2019, 6:34am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945 "2019-04-29T06:34:29Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [April 29, 2019, 6:34am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/1 "2019-04-29T06:34:29Z")
Warm greetings to Hail community. gwas = hl.linear\_regression\_rows(y=mt.pheno.CaffeineConsumption, x=mt.GT.n\_alt\_alleles(), covariates=[1.0]) While I was running the above command in jupyter notebook in centos 7 , I got the following error:- > \*\*019-04-29 11:38:35 Hail: INFO: linear\_regression\_rows: running on 250 samples for 1 response variable y, > with input variable x, and 1 additional covariate… > ERROR:root:Exception while sending command. > Traceback (most recent call last): > File “/home/aby/spark/python/lib/py4j-0.10.7-src.zip/py4j/java\_gateway.py”, line 1159, in send\_command > raise Py4JNetworkError(“Answer from Java side is empty”) > py4j.protocol.Py4JNetworkError: Answer from Java side is empty > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > File “/home/aby/spark/python/lib/py4j-0.10.7-src.zip/py4j/java\_gateway.py”, line 985, in send\_command > response = connection.send\_command(command) > File “/home/aby/spark/python/lib/py4j-0.10.7-src.zip/py4j/java\_gateway.py”, line 1164, in send\_command > “Error while receiving”, e, proto.ERROR\_ON\_RECEIVE) > 4j.protocol.Py4JNetworkError: Error while receiving Initially I thought this might be because of Java home is not available, but I checked and set my JAVA\_HOME variable , still I am getting the same error. Below is my JAVA\_HOME variable’s Output. \*\*[root@localhost ~]# echo $JAVA\_HOME /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7\_6.x86\_64/jre/ \*\* Kindly help me in resolving the issue. ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [April 29, 2019, 1:46pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/2 "2019-04-29T13:46:00Z")
what is the full version (with git SHA) of Hail you’re using? ---
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [May 1, 2019, 5:05am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/3 "2019-05-01T05:05:58Z")
> > > import hail > > > import hail as hl > > > hl.init() > > > 19/05/01 10:33:12 WARN Utils: Your hostname, localhost.localdomain resolves to a loopback address: 127.0.0.1; using 10.149.240.149 instead (on interface wlp8s0) > > > 19/05/01 10:33:12 WARN Utils: Set SPARK\_LOCAL\_IP if you need to bind to another address > > > 19/05/01 10:33:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable > > > Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties > > > Setting default log level to “WARN”. > > > To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). > > > Running on Apache Spark version 2.4.1 > > > SparkUI available at [http://10.149.240.149:4040](http://10.149.240.149:4040) > > > Welcome to > > > \_\_ \_\_ \<\>\_\_ > > > / /_/ /\_\_ \_\_/ / > > > / \_\_ / \_ `/ / / > > > /_/ /_/\_,_/_/_/ version 0.2.12-2571917c39c6 > > > LOGGING: writing to /root/hail-20190501-1033-0.2.12-2571917c39c6.log The hail version is 0.2.12-2571917c39c6. ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [May 1, 2019, 12:58pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/4 "2019-05-01T12:58:53Z")
Is there any java stack trace? ---
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [May 3, 2019, 9:03am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/5 "2019-05-03T09:03:25Z")
I am attaching the backend terminal output of jupyter notebook apart from this I didnt find any stack trace for java. **[I 14:26:23.396 NotebookApp] 302 GET /?token=20c3fe169bcd06b8b53b7da8a5f35092f8c03297a929411b (::1) 0.74ms** **[I 14:26:30.069 NotebookApp] Kernel started: d75dedfd-1718-44d9-be5c-4cde788f528e** **[I 14:26:32.806 NotebookApp] Adapting to protocol v5.1 for kernel d75dedfd-1718-44d9-be5c-4cde788f528e** **19/05/03 14:26:44 WARN Utils: Your hostname, localhost.localdomain resolves to a loopback address: 127.0.0.1; using 10.149.240.149 instead (on interface wlp8s0)** **19/05/03 14:26:44 WARN Utils: Set SPARK\_LOCAL\_IP if you need to bind to another address** **19/05/03 14:26:46 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable** **Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties** **Setting default log level to “WARN”.** **To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).** **[Stage 32:\> (0 + 2) / 2]/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7\_6.x86\_64/jre//bin/java: symbol lookup error: /tmp/jniloader8933272701730098438netlib-native\_system-linux-x86\_64.so: undefined symbol: cblas\_dgemm** **[I 14:28:29.992 NotebookApp] Saving file at /gwas.ipynb** ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [May 4, 2019, 7:52pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/6 "2019-05-04T19:52:00Z")
I think you need to install openblas ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [June 10, 2019, 3:04pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/7 "2019-06-10T15:04:51Z")
This error can occur on linux systems for which Hail doesn’t precomile native libraries, like centos. The solution is to run: `./gradlew releaseJar` instead of `./gradlew shadowJar` to build the jar – this will compile the native libraries as well. Note that the mechanisms for building Hail are subject to change. ---
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [July 3, 2019, 10:04am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/8 "2019-07-03T10:04:36Z")
Hello tpoterba, I tried using ./gradlew releaseJar instead of ./gradlew shadowJar and got the following error:- _ **g++ -o build/ibs.o -march=sandybridge -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-aliasing -I…/resources/include -I/usr/lib/jvm/java/include -I/usr/lib/jvm/java/include/linux -MD -MF build/ibs.d -MT build/ibs.o -c ibs.cpp** _ _ **In file included from ibs.cpp:5:0:** _ _ **ibs.h:17:10: fatal error: simdpp/simd.h: No such file or directory** _ \*\*\* #include “simdpp/simd.h”\*\*\* \*\*\* ^ ~~~~~~~~~~~~~~ \*\*\* _ **compilation terminated.** _ _**make: \*\*\* [build/ibs.o] Error 1**_ _ **FAILURE: Build failed with an exception.** _ \*\*\*\* What went wrong:\*\*\* _ **Execution failed for task ‘:nativeLibPrebuilt’.** _ _ **\> Process ‘command ‘make’’ finished with non-zero exit value 2** _ One more point is that I have succesfully installed hail in my machine without compiling it from scratch, still I am facing py4j.protocol.py4jNetworkError issue when trying to run linear regression model. So if you can point out how to solve this python and java connectivity issue, that will be really helpful. Thank you Regards aby ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [July 8, 2019, 3:09am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/9 "2019-07-08T03:09:23Z")
The build system should addressed this for you, but you’re missing libsimdpp, try: cd src/main/c make libsimdpp-2.1 Then retry ./gradlew releaseJar ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [July 8, 2019, 3:10am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/10 "2019-07-08T03:10:46Z")
Can you share the full make log? I want to understand why the build system didn’t automatically download libsimdpp. ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [July 8, 2019, 3:11am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/11 "2019-07-08T03:11:39Z")
With respect to the PY4J issue, please share the full error message and the hail log file. ---
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [July 12, 2019, 7:13am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/12 "2019-07-12T07:13:23Z")
Dear danking, I am attaching the complete py4j error:- > 2019-07-12 12:16:01 Hail: INFO: linear\_regression\_rows: running on 284 samples for 1 response variable y, > with input variable x, and 1 additional covariate… > ERROR:root:Exception while sending command. > Traceback (most recent call last): > File “/usr/lib/python3.6/site-packages/py4j/java\_gateway.py”, line 1159, in send\_command > raise Py4JNetworkError(“Answer from Java side is empty”) > py4j.protocol.Py4JNetworkError: Answer from Java side is empty > > During handling of the above exception, another exception occurred: > > ## Traceback (most recent call last): File “/usr/lib/python3.6/site-packages/py4j/java\_gateway.py”, line 985, in send\_command response = connection.send\_command(command) File “/usr/lib/python3.6/site-packages/py4j/java\_gateway.py”, line 1164, in send\_command “Error while receiving”, e, proto.ERROR\_ON\_RECEIVE) py4j.protocol.Py4JNetworkError: Error while receiving > > Py4JError Traceback (most recent call last) > in > 1 gwas = hl.linear\_regression\_rows(y=mt.pheno.CaffeineConsumption, > 2 x=mt.GT.n\_alt\_alleles(), > ----\> 3 covariates=[1.0]) > > \ in linear\_regression\_rows(y, x, covariates, block\_size, pass\_through) > > /usr/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(\_\_original\_func, \*args, \*\*kwargs) > 559 def wrapper(_original\_func, \*args, \*\*kwargs): > 560 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method) > → 561 return _original\_func(\*args, \*\*kwargs_) > 562 > 563 return wrapper > > /usr/lib/python3.6/site-packages/hail/methods/statgen.py in linear\_regression\_rows(y, x, covariates, block\_size, pass\_through) > 435 ht\_result = ht\_result.annotate(\*\*{f: ht\_result[f][0] for f in fields}) > 436 > → 437 return ht\_result.persist() > 438 > 439 > > \ in persist(self, storage\_level) > > /usr/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(\_\_original\_func, \*args, \*\*kwargs) > 559 def wrapper(_original\_func, \*args, \*\*kwargs): > 560 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method) > → 561 return _original\_func(\*args, \*\*kwargs_) > 562 > 563 return wrapper > > /usr/lib/python3.6/site-packages/hail/table.py in persist(self, storage\_level) > 1757 Persisted table. > 1758 “”" > → 1759 return Env.backend().persist\_table(self, storage\_level) > 1760 > 1761 def unpersist(self) → ‘Table’: > > /usr/lib/python3.6/site-packages/hail/backend/backend.py in persist\_table(self, t, storage\_level) > 123 > 124 def persist\_table(self, t, storage\_level): > → 125 return Table.\_from\_java(self.\_to\_java\_ir(t.\_tir).pyPersist(storage\_level)) > 126 > 127 def unpersist\_table(self, t): > > /usr/lib/python3.6/site-packages/py4j/java\_gateway.py in **call** (self, \*args) > 1255 answer = self.gateway\_client.send\_command(command) > 1256 return\_value = get\_return\_value( > → 1257 answer, self.gateway\_client, self.target\_id, self.name) > 1258 > 1259 for temp\_arg in temp\_args: > > /usr/lib/python3.6/site-packages/hail/utils/java.py in deco(\*args, \*\*kwargs) > 226 import pyspark > 227 try: > → 228 return f(\*args, \*\*kwargs) > 229 except py4j.protocol.Py4JJavaError as e: > 230 s = e.java\_exception.toString() > > /usr/lib/python3.6/site-packages/py4j/protocol.py in get\_return\_value(answer, gateway\_client, target\_id, name) > 334 raise Py4JError( > 335 “An error occurred while calling {0}{1}{2}”. > → 336 format(target\_id, “.”, name)) > 337 else: > 338 type = answer[1] > > Py4JError: An error occurred while calling o149.pyPersist ​ ---
**Author:** ![aby](https://avatars.discourse-cdn.com/v4/letter/a/a88e57/32.png) [@aby](https://discuss.hail.is/u/aby)\ **Post date:** [July 12, 2019, 7:27am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/14 "2019-07-12T07:27:30Z")
Dear danking, The hail log file is available in the following link:- [https://drive.google.com/file/d/1HhwLI0Ay8JlziJlCqHZ7Cg0BoCcyJ9eJ/view?usp=sharing](https://drive.google.com/file/d/1HhwLI0Ay8JlziJlCqHZ7Cg0BoCcyJ9eJ/view?usp=sharing) ---
**Author:** ![wkl1990](https://avatars.discourse-cdn.com/v4/letter/w/7ea924/32.png) [@wkl1990](https://discuss.hail.is/u/wkl1990)\ **Post date:** [October 25, 2019, 7:11am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/15 "2019-10-25T07:11:14Z")
I meet the same error. Have you solved it and how? ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [October 25, 2019, 10:03am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/16 "2019-10-25T10:03:56Z")
You’re getting this error from linear regression rows in Hail as well? ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [October 25, 2019, 3:40pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/17 "2019-10-25T15:40:51Z")
I suspect this is a BLAS issue. @wkl1990, what is your compute environment? Do you use Google Dataproc, Amazon EMR, an on-premises cluster, or something else? Do you have BLAS installed on your compute nodes? What is the OS and CPU architecture on your compute nodes and on your leader node? ---
**Author:** ![wkl1990](https://avatars.discourse-cdn.com/v4/letter/w/7ea924/32.png) [@wkl1990](https://discuss.hail.is/u/wkl1990)\ **Post date:** [October 28, 2019, 7:35am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/18 "2019-10-28T07:35:49Z")
I installed hail on my local server. The OS is Linux version 3.10.0-514.10.2.el7.x86\_64 (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ). The CPU is CPU0: Intel® Xeon® CPU E7-8895 v3 @ 2.60GHz. I found the blas installed (blas-3.4.2-4.el7.x86\_64). ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [October 28, 2019, 2:13pm UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/19 "2019-10-28T14:13:54Z")
You probably need to compile hail from source. I think we build the pip hail for newer CPUs. Are you able to download the source and compile from source? ---
**Author:** ![wkl1990](https://avatars.discourse-cdn.com/v4/letter/w/7ea924/32.png) [@wkl1990](https://discuss.hail.is/u/wkl1990)\ **Post date:** [October 30, 2019, 7:04am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/20 "2019-10-30T07:04:13Z")
I downloaded the source of hail from github and installed successfully using “make install”. However, I still got the same error when “run linear\_regression\_rows”. ---
**Author:** ![Nitha](https://avatars.discourse-cdn.com/v4/letter/n/eb9ed0/32.png) [@Nitha](https://discuss.hail.is/u/Nitha)\ **Post date:** [October 30, 2019, 8:24am UTC](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945/21 "2019-10-30T08:24:40Z")
Hi All, Could anyone point out the solution for the error discussed (question asked by @aby: [py4j.protocol.Py4JNetworkError: Answer from Java side is empty](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945)? Still, we are stuck in this error, we had tried all the possibilities, but still it persists. Does anyone have a solution for this? Thanks a lot! [Next page](https://discuss.hail.is/t/py4j-protocol-py4jnetworkerror-answer-from-java-side-is-empty/945.md?page=2)