Answer from Java side is empty

[DAU]

I caught such error when I tried to write gwas results:

gwas.write(‘ldlp.ht’)

Hail version is 0.2.65-367cf1874d85

I used:

use OpenBLAS
use .openblas_for_hail-0.2.20
export LD_PRELOAD=/broad/software/free/Linux/redhat_7_x86_64/pkgs/openblas_0.2.20/lib/libopenblas.so
but It does not help
I am working on Broad cluster and when I run it in jupyter based on Anaconda3 all works well

Exception happened during processing of request from (‘127.0.0.1’, 44310)
ERROR:root:Exception while sending command.
Traceback (most recent call last):
File “/humgen/atgu1/methods/dusoltsev/programms/hail_3_8/lib/python3.8/site-packages/py4j/java_gateway.py”, line 1207, in send_command
raise Py4JNetworkError(“Answer from Java side is empty”)
py4j.protocol.Py4JNetworkError: Answer from Java side is empty

[tpoterba]

This looks like the JVM is running out of memory. Can you try setting the PYSPARK_SUBMIT_ARGS env var as in BGEN filter_cols memory leak?

@ebartell

export PYSPARK_SUBMIT_ARGS=“–driver-memory 8G pyspark-shell --executor-memory 8G” ## and with 4G as well. Without this line, I run out of memory a la “java.lang.OutOfMemoryError: Java heap space, in Stage 2.”