# Java error in Jupyter notebook on google cloud

**URL:** <https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816>\
**Category:** Hail Query & hailctl\
**Created:** [November 27, 2020, 4:47am UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816 "2020-11-27T04:47:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![lilliank](https://avatars.discourse-cdn.com/v4/letter/l/ecc23a/32.png) [@lilliank](https://discuss.hail.is/u/lilliank)\
**Post date:** [November 27, 2020, 4:47am UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816/1 "2020-11-27T04:47:09Z")

</div>

Hello,

I am getting an error when I run the following command:

```auto
ukbb.write('file_path', overwrite=True)

```

The error is posted below. When I ran the same code yesterday with the same set up I didn’t get an error, but I am now working with a larger dataset. Thanks!

```auto
ERROR:root:Exception while sending command.
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1159, in send_command
    raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 985, in send_command
    response = connection.send_command(command)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1164, in send_command
    "Error while receiving", e, proto.ERROR_ON_RECEIVE)
py4j.protocol.Py4JNetworkError: Error while receiving
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-7-fbd959e44a2b>", line 8, in <module>
    metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py", line 2426, in count
    return Env.backend().execute(count_ir)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 74, in execute
    result = json.loads(self._jhc.backend().executeJSON(jir))
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py", line 16, in deco
    return f(*args, **kwargs)
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 336, in get_return_value
    format(target_id, ".", name))
py4j.protocol.Py4JError: An error occurred while calling o164.executeJSON

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/miniconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2044, in showtraceback
    stb = value._render_traceback_()
AttributeError: 'Py4JError' object has no attribute '_render_traceback_'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
---------------------------------------------------------------------------
Py4JError Traceback (most recent call last)
<ipython-input-7-fbd959e44a2b> in <module>
      6 while(True):
      7 if not path.exists("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt"):
----> 8 metadata.to_csv("gs://ukb-gt/ukbQC_"+str(ukbb.count()[0])+"_x_"+str(ukbb.count()[1])+"_metadata"+str(imeta)+".txt", index=False)
      9 break
     10 

/opt/conda/miniconda3/lib/python3.6/site-packages/hail/matrixtable.py in count(self)
   2424 """
   2425 count_ir = ir.MatrixCount(self._mir)
-> 2426 return Env.backend().execute(count_ir)
   2427 
   2428 @typecheck_method(output=str,

/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py in execute(self, ir, timed)
     72 # print(self._hail_package.expr.ir.Pretty.apply(jir, True, -1))
     73 try:
---> 74 result = json.loads(self._jhc.backend().executeJSON(jir))
     75 value = ir.typ._from_json(result['value'])
     76 timings = result['timings']

/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __call__ (self, *args)
   1255 answer = self.gateway_client.send_command(command)
   1256 return_value = get_return_value(
-> 1257 answer, self.gateway_client, self.target_id, self.name)
   1258 
   1259 for temp_arg in temp_args:

/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/py4j_backend.py in deco(*args, **kwargs)
     14 import pyspark
     15 try:
---> 16 return f(*args, **kwargs)
     17 except py4j.protocol.Py4JJavaError as e:
     18 s = e.java_exception.toString()

/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
    334 raise Py4JError(
    335 "An error occurred while calling {0}{1}{2}".
--> 336 format(target_id, ".", name))
    337 else:
    338 type = answer[1]

Py4JError: An error occurred while calling o164.executeJSON
'''
```

---

<div class="post-metadata">

**Author:** ![lilliank](https://avatars.discourse-cdn.com/v4/letter/l/ecc23a/32.png) [@lilliank](https://discuss.hail.is/u/lilliank)\
**Post date:** [November 28, 2020, 6:21pm UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816/2 "2020-11-28T18:21:46Z")

</div>

I tried updating hail and gcloud using:

```auto
pip3 install -U hail
gcloud components update

```

Now I get the following error whenever I try to read or write to any file:

```auto
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35125)
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in _get_connection(self)
    928 try:
--> 929 connection = self.deque.pop()
    930 except IndexError:
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
ConnectionRefusedError Traceback (most recent call last)
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in start(self)
   1066 try:
-> 1067 self.socket.connect((self.address, self.port))
   1068 self.stream = self.socket.makefile("rb")
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Py4JNetworkError Traceback (most recent call last)
<ipython-input-23-586fb829c89f> in <module>
      1 # Define phenotype and assessment centre classification:
----> 2 withdrawn = hl.read_table(withdrawn_file)
      3 gwassamples = hl.read_table(gwassamples_file)
<decorator-gen-1439> in read_table(path, _intervals, _filter_intervals)
/opt/conda/miniconda3/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    612 def wrapper(__original_func, *args, **kwargs):
    613 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 614 return __original_func(*args_, **kwargs_)
    615 
    616 return wrapper
/opt/conda/miniconda3/lib/python3.6/site-packages/hail/methods/impex.py in read_table(path, _intervals, _filter_intervals)
   2436 :class:`.Table`
   2437 """
-> 2438 for rg_config in Env.backend().load_references_from_dataset(path):
   2439 hl.ReferenceGenome._from_config(rg_config)
   2440 
/opt/conda/miniconda3/lib/python3.6/site-packages/hail/backend/spark_backend.py in load_references_from_dataset(self, path)
    320 
    321 def load_references_from_dataset(self, path):
--> 322 return json.loads(Env.hail().variant.ReferenceGenome.fromHailDataset(self.fs._jfs, path))
    323 
    324 def from_fasta_file(self, name, fasta_file, index_file, x_contigs, y_contigs, mt_contigs, par):
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in __getattr__ (self, name)
   1589 self._jvm_id + "\n" +\
   1590 proto.END_COMMAND_PART
-> 1591 answer = self._gateway_client.send_command(command)
   1592 if answer == proto.SUCCESS_PACKAGE:
   1593 return JavaPackage(new_fqn, self._gateway_client, self._jvm_id)
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in send_command(self, command, retry, binary)
    981 if `binary` is `True`.
    982 """
--> 983 connection = self._get_connection()
    984 try:
    985 response = connection.send_command(command)
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in _get_connection(self)
    929 connection = self.deque.pop()
    930 except IndexError:
--> 931 connection = self._create_connection()
    932 return connection
    933 
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in _create_connection(self)
    935 connection = GatewayConnection(
    936 self.gateway_parameters, self.gateway_property)
--> 937 connection.start()
    938 return connection
    939 
/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py in start(self)
   1077 "server ({0}:{1})".format(self.address, self.port)
   1078 logger.exception(msg)
-> 1079 raise Py4JNetworkError(msg, e)
   1080 
   1081 def _authenticate_connection(self):
Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:35125)

```

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [November 29, 2020, 2:48pm UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816/3 "2020-11-29T14:48:26Z")

</div>

Can you share the hail log file? That has the true error.

First guess: missing linear algebra libraries. Did you apt get install openblas?

---

<div class="post-metadata">

**Author:** ![lilliank](https://avatars.discourse-cdn.com/v4/letter/l/ecc23a/32.png) [@lilliank](https://discuss.hail.is/u/lilliank)\
**Post date:** [November 29, 2020, 7:55pm UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816/4 "2020-11-29T19:55:21Z")

</div>

I don’t think I can create a log file without getting the error.

I tried the following:

```auto
import hail as hl
from hail.plot import show
from os import path
import os
import pandas as pd
hl.plot.output_notebook()

os.system('apt get install openblas')

hl.init(log='gs://ukb-gt/log.log')

```

But I still get the same error when I get to the last line of setting the log file. I also cannot find any log file in gs://ukb-gt/

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [November 30, 2020, 2:58pm UTC](https://discuss.hail.is/t/java-error-in-jupyter-notebook-on-google-cloud/1816/5 "2020-11-30T14:58:22Z")

</div>

You can’t log to a google cloud storage bucket, you have to log to a local file. A log will automatically be created by Hail when it initializes. It will print the log location after the Hail logo. It should look like this:

```bash
Welcome to
     ____ <>__
    / /_/ / ____ / /
   / __ / _ `/ / /
  /_/ /_/\_,_/_/_/ version 0.2.60-de1845e1c2f6
LOGGING: writing to /Users/dking/projects/hail/hail-20201130-0957-0.2.60-de1845e1c2f6.log

```

EDIT: Also, does `apt get` actually work? It’s been a while since I was a daily linux user but I thought it had to be `apt-get`. (Typo mine in my previous post, apologies)