# Java server breaks only when running SKAT

**URL:** https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282
**Category:** Hail Query & hailctl
**Created:** [February 11, 2020, 12:34am UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282 "2020-02-11T00:34:01Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 11, 2020, 12:34am UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/1 "2020-02-11T00:34:01Z")

</div>

I am running hail 0.2 from conda, on HPC (centos, slurm, 1 node 20 CPU 16GB per CPU).  
Whenever I try to run SKAT (and not on any other function), the java server breaks and I need to initialize hail again.

`ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35675)`

My code:  
`skat_table = hl.skat(key_expr=chr21f.genes2, weight_expr=(hl.dbeta(hl.min(chr21f.variant_qc.AF), 1.0, 25.0) ** 2), y=chr21f.sys_bool, x=chr21f.GT.n_alt_alleles(), covariates=[1]) `  
Error:

```
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35675)
Traceback (most recent call last):
  File "/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _get_connection(self)
    928 try:
--> 929 connection = self.deque.pop()
    930 except IndexError:

IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

ConnectionRefusedError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in start(self)
   1066 try:
-> 1067 self.socket.connect((self.address, self.port))
   1068 self.stream = self.socket.makefile("rb")

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Py4JNetworkError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/IPython/core/formatters.py in __call__ (self, obj)
    700 type_pprinters=self.type_printers,
    701 deferred_pprinters=self.deferred_printers)
--> 702 printer.pretty(obj)
    703 printer.flush()
    704 return stream.getvalue()

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    398 if cls is not object \
    399 and callable(cls. __dict__.get(' __repr__')):
--> 400 return _repr_pprint(obj, self, cycle)
    401 
    402 return _default_pprint(obj, self, cycle)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    693 """A pprint that just redirects to the normal repr function."""
    694 # Find newlines and replace them with p.break_()
--> 695 output = repr(obj)
    696 for idx,output_line in enumerate(output.splitlines()):
    697 if idx:

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in __repr__ (self)
   1242 
   1243 def __repr__ (self):
-> 1244 return self. __str__ ()
   1245 
   1246 def data(self):

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in __str__ (self)
   1239 
   1240 def __str__ (self):
-> 1241 return self._ascii_str()
   1242 
   1243 def __repr__ (self):

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in _ascii_str(self)
   1266 return s
   1267 
-> 1268 rows, has_more, dtype = self.data()
   1269 fields = list(dtype)
   1270 trunc_fields = [trunc(f) for f in fields]

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in data(self)
   1249 row_dtype = t.row.dtype
   1250 t = t.select(**{k: Table._hl_format(v, self.truncate) for (k, v) in t.row.items()})
-> 1251 rows, has_more = t._take_n(self.n)
   1252 self._data = (rows, has_more, row_dtype)
   1253 return self._data

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in _take_n(self, n)
   1370 has_more = False
   1371 else:
-> 1372 rows = self.take(n + 1)
   1373 has_more = len(rows) > n
   1374 rows = rows[:n]

<decorator-gen-1054> in take(self, n, _localize)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    583 def wrapper(__original_func, *args, **kwargs):
    584 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 585 return __original_func(*args_, **kwargs_)
    586 
    587 return wrapper

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in take(self, n, _localize)
   2062 """
   2063 
-> 2064 return self.head(n).collect(_localize)
   2065 
   2066 @typecheck_method(n=int)

<decorator-gen-1048> in collect(self, _localize)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    583 def wrapper(__original_func, *args, **kwargs):
    584 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 585 return __original_func(*args_, **kwargs_)
    586 
    587 return wrapper

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in collect(self, _localize)
   1861 e = construct_expr(ir, hl.tarray(t.row.dtype))
   1862 if _localize:
-> 1863 return Env.backend().execute(e._ir)
   1864 else:
   1865 return e

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/backend/backend.py in execute(self, ir, timed)
    107 
    108 def execute(self, ir, timed=False):
--> 109 result = json.loads(Env.hc()._jhc.backend().executeJSON(self._to_java_ir(ir)))
    110 value = ir.typ._from_json(result['value'])
    111 timings = result['timings']

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in __call__ (self, *args)
   1253 proto.END_COMMAND_PART
   1254 
-> 1255 answer = self.gateway_client.send_command(command)
   1256 return_value = get_return_value(
   1257 answer, self.gateway_client, self.target_id, self.name)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in send_command(self, command, retry, binary)
    981 if `binary` is `True`.
    982 """
--> 983 connection = self._get_connection()
    984 try:
    985 response = connection.send_command(command)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _get_connection(self)
    929 connection = self.deque.pop()
    930 except IndexError:
--> 931 connection = self._create_connection()
    932 return connection
    933 

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _create_connection(self)
    935 connection = GatewayConnection(
    936 self.gateway_parameters, self.gateway_property)
--> 937 connection.start()
    938 return connection
    939 

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in start(self)
   1077 "server ({0}:{1})".format(self.address, self.port)
   1078 logger.exception(msg)
-> 1079 raise Py4JNetworkError(msg, e)
   1080 
   1081 def _authenticate_connection(self):

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:35675)

ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:35675)
Traceback (most recent call last):
  File "/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py", line 929, in _get_connection
    connection = self.deque.pop()
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py", line 1067, in start
    self.socket.connect((self.address, self.port))
ConnectionRefusedError: [Errno 111] Connection refused
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _get_connection(self)
    928 try:
--> 929 connection = self.deque.pop()
    930 except IndexError:

IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

ConnectionRefusedError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in start(self)
   1066 try:
-> 1067 self.socket.connect((self.address, self.port))
   1068 self.stream = self.socket.makefile("rb")

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Py4JNetworkError Traceback (most recent call last)
/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/IPython/core/formatters.py in __call__ (self, obj)
    343 method = get_real_method(obj, self.print_method)
    344 if method is not None:
--> 345 return method()
    346 return None
    347 else:

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in _repr_html_(self)
   1254 
   1255 def _repr_html_(self):
-> 1256 return self._html_str()
   1257 
   1258 def _ascii_str(self):

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in _html_str(self)
   1340 types = self.types
   1341 
-> 1342 rows, has_more, dtype = self.data()
   1343 fields = list(dtype)
   1344 

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in data(self)
   1249 row_dtype = t.row.dtype
   1250 t = t.select(**{k: Table._hl_format(v, self.truncate) for (k, v) in t.row.items()})
-> 1251 rows, has_more = t._take_n(self.n)
   1252 self._data = (rows, has_more, row_dtype)
   1253 return self._data

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in _take_n(self, n)
   1370 has_more = False
   1371 else:
-> 1372 rows = self.take(n + 1)
   1373 has_more = len(rows) > n
   1374 rows = rows[:n]

<decorator-gen-1054> in take(self, n, _localize)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    583 def wrapper(__original_func, *args, **kwargs):
    584 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 585 return __original_func(*args_, **kwargs_)
    586 
    587 return wrapper

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in take(self, n, _localize)
   2062 """
   2063 
-> 2064 return self.head(n).collect(_localize)
   2065 
   2066 @typecheck_method(n=int)

<decorator-gen-1048> in collect(self, _localize)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    583 def wrapper(__original_func, *args, **kwargs):
    584 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 585 return __original_func(*args_, **kwargs_)
    586 
    587 return wrapper

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/table.py in collect(self, _localize)
   1861 e = construct_expr(ir, hl.tarray(t.row.dtype))
   1862 if _localize:
-> 1863 return Env.backend().execute(e._ir)
   1864 else:
   1865 return e

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/hail/backend/backend.py in execute(self, ir, timed)
    107 
    108 def execute(self, ir, timed=False):
--> 109 result = json.loads(Env.hc()._jhc.backend().executeJSON(self._to_java_ir(ir)))
    110 value = ir.typ._from_json(result['value'])
    111 timings = result['timings']

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in __call__ (self, *args)
   1253 proto.END_COMMAND_PART
   1254 
-> 1255 answer = self.gateway_client.send_command(command)
   1256 return_value = get_return_value(
   1257 answer, self.gateway_client, self.target_id, self.name)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in send_command(self, command, retry, binary)
    981 if `binary` is `True`.
    982 """
--> 983 connection = self._get_connection()
    984 try:
    985 response = connection.send_command(command)

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _get_connection(self)
    929 connection = self.deque.pop()
    930 except IndexError:
--> 931 connection = self._create_connection()
    932 return connection
    933 

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in _create_connection(self)
    935 connection = GatewayConnection(
    936 self.gateway_parameters, self.gateway_property)
--> 937 connection.start()
    938 return connection
    939 

/gpfs/share/apps/anaconda3/cpu/5.2.0/lib/python3.6/site-packages/py4j/java_gateway.py in start(self)
   1077 "server ({0}:{1})".format(self.address, self.port)
   1078 logger.exception(msg)
-> 1079 raise Py4JNetworkError(msg, e)
   1080 
   1081 def _authenticate_connection(self):

Py4JNetworkError: An error occurred while trying to connect to the Java server (127.0.0.1:35675)
```

---

<div class="post-metadata">

### Author: ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [February 11, 2020, 1:48pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/2 "2020-02-11T13:48:41Z")

</div>

This is probably a problem with the C libraries. Is there any more information in the Hail log? Is there a file in the hail working directory of the driver/workers that looks like `hs_err_pid...`?

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 11, 2020, 5:01pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/3 "2020-02-11T17:01:31Z")

</div>

No- there are only the hail log files.

Thanks

---

<div class="post-metadata">

### Author: ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)
#### Post date: [February 12, 2020, 12:31am UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/4 "2020-02-12T00:31:11Z")

</div>

Because you are running on centos & probably a different CPU architecture than GCP Dataproc nodes, you need to compile hail from source. See details [here](https://hail.is/docs/0.2/getting_started.html). Basically:

```auto
git clone https://github.com/hail-is/hail.git
cd hail/hail
make install HAIL_COMPILE_NATIVES=1

```

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 12, 2020, 9:10pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/5 "2020-02-12T21:10:24Z")

</div>

thanks Dan, I asked out HPC team to install compile it (since I am not a cluster sudo), and they did so but I still get the same error right when trying to show the skat.table (when the skat expression is starting to compute), followed by loss of functionality of hail (until re init).

```
    2020-02-12 16:05:52 Hail: WARN: 42 of 3315 samples have a missing phenotype or covariate.
ERROR:root:Exception while sending command.
Traceback (most recent call last):
  File "/gpfs/share/apps/python/cpu/3.6.5/lib/python3.6/site-packages/py4j/java_gateway.py", line 1159, in send_command
    raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/gpfs/share/apps/python/cpu/3.6.5/lib/python3.6/site-packages/py4j/java_gateway.py", line 985, in send_command
    response = connection.send_command(command)
  File "/gpfs/share/apps/python/cpu/3.6.5/lib/python3.6/site-packages/py4j/java_gateway.py", line 1164, in send_command
    "Error while receiving", e, proto.ERROR_ON_RECEIVE)
py4j.protocol.Py4JNetworkError: Error while receiving
```

---

<div class="post-metadata">

### Author: ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)
#### Post date: [February 12, 2020, 9:16pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/6 "2020-02-12T21:16:10Z")

</div>

Can you attach the hail log file? It’s location is printed near the message that has the ASCII art of “hail”.

---

<div class="post-metadata">

### Author: ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)
#### Post date: [February 13, 2020, 3:10pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/7 "2020-02-13T15:10:31Z")

</div>

The spark log ends abruptly upon loading the native libraries. Is there any files with names containing `hs_err_pid`? Are you certain hail was recompiled with the above command? In particular, you need `HAIL_COMPILE_NATIVE=1` after the `install`.

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 13, 2020, 3:19pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/8 "2020-02-13T15:19:44Z")

</div>

Thanks Dan.  
No - no hs\_err\_pid file is generated at the working directory or its subdirectories.  
I will ask about they compile native.

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 13, 2020, 3:38pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/9 "2020-02-13T15:38:59Z")

</div>

Yes he compiled it with that option. Is the hs\_err\_pid supposed to be located in the working directory or in the installation directory?

---

<div class="post-metadata">

### Author: ![johnc1231](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/johnc1231/32/286_2.png) [@johnc1231](https://discuss.hail.is/u/johnc1231)
#### Post date: [February 13, 2020, 3:45pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/10 "2020-02-13T15:45:29Z")

</div>

It would appear in the working directory.

---

<div class="post-metadata">

### Author: ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)
#### Post date: [February 13, 2020, 3:55pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/11 "2020-02-13T15:55:52Z")

</div>

@orr This is strongly suggestive that the native libraries are still not installed properly.

Let’s create a python virtual environment, install hail into that (no need for sudo) and verify it is still broken there:

```auto
python3 -m venv hail-natives-test
source hail-natives-test/bin/activate
cd /path/to/hail/repo
make install HAIL_COMPILE_NATIVES=1

```

then let’s try a very simple example:

```auto
mt = hl.balding_nichols_model(3, 10, 10)
mt = mt.annotate_cols(pheno = hl.rand_unif(0, 1))
mt = mt.annotate_rows(gene = mt.locus.position // 3)
mt = mt.annotate_rows(weight = hl.rand_unif(0, 1))
hl.skat(key_expr=mt.gene,
        weight_expr=mt.weight,
        y=mt.pheno,
        x=mt.GT.n_alt_alleles(),
        covariates=[]
).show()

```

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 14, 2020, 8:15pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/12 "2020-02-14T20:15:09Z")

</div>

After:  
`make install HAIL_COMPILE_NATIVES=1`  
I get:

```
SHORT_REVISION is set to "8edccade0e51" which is different from old value ""
printf "8edccade0e51" > env/SHORT_REVISION
HAIL_PIP_VERSION is set to "0.2.32" which is different from old value ""
printf "0.2.32" > env/HAIL_PIP_VERSION
echo 0.2.32-8edccade0e51 > python/hail/hail_version
echo 0.2.32 > python/hail/hail_pip_version
cp -f python/hail/hail_version python/hailtop/hailctl/hail_version
printf 'hail_version="0.2.32-8edccade0e51";' > python/hail/docs/_static/hail_version.js
printf 'hail_pip_version="0.2.32"' >> python/hail/docs/_static/hail_version.js
REVISION is set to "8edccade0e51a49028f8bd29a3b3209464b84c08" which is different from old valu$
 ""
printf "8edccade0e51a49028f8bd29a3b3209464b84c08" > env/REVISION
BRANCH is set to "master" which is different from old value ""
printf "master" > env/BRANCH
URL is set to "https://github.com/hail-is/hail.git" which is different from old value ""
printf "https://github.com/hail-is/hail.git" > env/URL
SPARK_VERSION is set to "2.4.0" which is different from old value ""
printf "2.4.0" > env/SPARK_VERSION
echo '[Build Metadata]' > src/main/resources/build-info.properties
echo 'user=yaacoo01' >> src/main/resources/build-info.properties
echo 'revision=8edccade0e51a49028f8bd29a3b3209464b84c08' >> src/main/resources/build-info.prop$
rties
echo 'branch=master' >> src/main/resources/build-info.properties
echo 'date=2020-02-14T20:08:48Z' >> src/main/resources/build-info.properties
echo 'url=https://github.com/hail-is/hail.git' >> src/main/resources/build-info.properties
echo 'sparkVersion=2.4.0' >> src/main/resources/build-info.properties
echo 'hailPipVersion=0.2.32' >> src/main/resources/build-info.properties
make -C src/main/c prebuilt
make[1]: Entering directory `/gpfs/scratch/yaacoo01/temp/hail/hail/src/main/c'
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux Upcalls.cpp -MG 
-M -MF build/Upcalls.d -MT build/Upcalls.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux testutils/unit-$
ests.cpp -MG -M -MF build/testutils/unit-tests.d -MT build/testutils/unit-tests.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux test.cpp -MG -M 
-MF build/test.d -MT build/test.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux Region_test.cpp 
-MG -M -MF build/Region_test.d -MT build/Region_test.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux Region.cpp -MG $
M -MF build/Region.d -MT build/Region.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC -ggdb -fno-strict-ali$
sing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/incl$
de -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/linux PartitionIterat$
rs.cpp -MG -M -MF build/PartitionIterators.d -MT build/PartitionIterators.o
g++: error: unrecognized command line option ‘-std=c++14’
g++ -o build/NativeBoot.o -march=corei7-avx -O3 -std=c++14 -Ilibsimdpp-2.1 -Wall -Wextra -fPIC 
-ggdb -fno-strict-aliasing -I../resources/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-1
1.b12.el7.x86_64/include -I/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/include/
linux -MD -MF build/NativeBoot.d -MT build/NativeBoot.o -c NativeBoot.cpp
g++: error: unrecognized command line option ‘-std=c++14’
make[1]: *** [build/NativeBoot.o] Error 1
make[1]: Leaving directory `/gpfs/scratch/yaacoo01/temp/hail/hail/src/main/c'
make: *** [native-lib-prebuilt] Error 2
```

---

<div class="post-metadata">

### Author: ![johnc1231](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/johnc1231/32/286_2.png) [@johnc1231](https://discuss.hail.is/u/johnc1231)
#### Post date: [February 14, 2020, 8:26pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/13 "2020-02-14T20:26:00Z")

</div>

> [@orr](#):
>
> g++: error: unrecognized command line option ‘-std=c++14’

My guess is that you have an old gcc version: [linux mint - c++: error: unrecognized command line option ‘-std=c++14’ - Stack Overflow](https://stackoverflow.com/questions/36245428/c-error-unrecognized-command-line-option-std-c14)

---

<div class="post-metadata">

### Author: ![johnc1231](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/johnc1231/32/286_2.png) [@johnc1231](https://discuss.hail.is/u/johnc1231)
#### Post date: [February 14, 2020, 8:26pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/14 "2020-02-14T20:26:47Z")

</div>

We require gcc 5.0 or later: [https://hail.is/docs/0.2/getting\_started.html](https://hail.is/docs/0.2/getting_started.html)

---

<div class="post-metadata">

### Author: ![orr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/orr/32/846_2.png) [@orr](https://discuss.hail.is/u/orr)
#### Post date: [February 14, 2020, 8:50pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/15 "2020-02-14T20:50:47Z")

</div>

Right, i loaded a gcc/6.1.0 module and no i get the following error:

```
Upcalls.cpp: In constructor ‘hail::UpcallConfig::UpcallConfig()’:
Upcalls.cpp:14:44: error: ‘JNI_VERSION_1_8’ was not declared in this scope
   auto rc = java_vm_->GetEnv((void**)&env, JNI_VERSION_1_8);
                                            ^ ~~~~~~~~~~~~~~
Upcalls.cpp: In constructor ‘hail::UpcallEnv::UpcallEnv()’:
Upcalls.cpp:69:39: error: ‘JNI_VERSION_1_8’ was not declared in this scope
   auto rc = vm->GetEnv((void**)&env_, JNI_VERSION_1_8);
                                       ^ ~~~~~~~~~~~~~~
make[1]: *** [build/Upcalls.o] Error 1
make[1]: Leaving directory `/gpfs/scratch/yaacoo01/temp/hail/hail/src/main/c'
make: *** [native-lib-prebuilt] Error 2
```

---

<div class="post-metadata">

### Author: ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)
#### Post date: [February 21, 2020, 9:59pm UTC](https://discuss.hail.is/t/java-server-breaks-only-when-running-skat/1282/16 "2020-02-21T21:59:12Z")

</div>

Are you certain you have JDK 8 installed?