Java Heap Space out of memory

You should set executor memory and driver memory in the same PYSPARK_SUBMIT_ARGS, like this:

export PYSPARK_SUBMIT_ARGS='--driver-memory 16g --executor-memory 16g pyspark-shell'
jupyter ...

That error usually means the real error is in the log file. Can you attach the log file here? The log file location is printed when Hail first starts.