Answer from Java side is empty

[tpoterba]

This looks like the JVM is running out of memory. Can you try setting the PYSPARK_SUBMIT_ARGS env var as in BGEN filter_cols memory leak?

@ebartell

export PYSPARK_SUBMIT_ARGS=“–driver-memory 8G pyspark-shell --executor-memory 8G” ## and with 4G as well. Without this line, I run out of memory a la “java.lang.OutOfMemoryError: Java heap space, in Stage 2.”