Fatal error at get_module_dir during installation example

Hail Query & hailctl
1

Hi,
I’m trying to install hail but without success, after reading several topics I run out of ideas.
Any help is welcome!

setup:

then I run the first example
ipython

import hail as hl
mt = hl.balding_nichols_model(n_populations=3, n_samples=50, n_variants=100)
mt.count()

I get:

(hail) vagrant@ubuntu-xenial:/share$ ipython
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 12:22:00)
Type ‘copyright’, ‘credits’ or ‘license’ for more information
IPython 7.5.0 – An enhanced Interactive Python. Type ‘?’ for help.

In [1]: import hail as hl                                                                                                                                                                                                                     

In [2]: mt = hl.balding_nichols_model(n_populations=3, n_samples=50, n_variants=100).count()                                                                                                                                                  
Initializing Spark and Hail with default parameters...
using hail jar at /home/vagrant/miniconda3/envs/hail/lib/python3.6/site-packages/hail/hail-all-spark.jar
19/07/03 12:50:16 WARN Utils: Your hostname, ubuntu-xenial resolves to a loopback address: 127.0.1.1; using 10.0.2.15 instead (on interface enp0s3)
19/07/03 12:50:16 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
19/07/03 12:50:17 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Running on Apache Spark version 2.4.1
SparkUI available at http://10.0.2.15:4040
Welcome to
     __  __     <>__
    / /_/ /__  __/ /
   / __  / _ `/ / /
  /_/ /_/\_,_/_/_/   version 0.2.16-6da0d3571629
LOGGING: writing to /share/hail-20190703-1250-0.2.16-6da0d3571629.log
2019-07-03 12:50:21 Hail: INFO: balding_nichols_model: generating genotypes for 3 populations, 50 samples, and 100 variants...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0x00007f267ade464b, pid=4007, tid=0x00007f268587f700
#
# JRE version: OpenJDK Runtime Environment (8.0_212-b03) (build 1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03)
# Java VM: OpenJDK 64-Bit Server VM (25.212-b03 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libhail8527984476743487864.so+0xd64b]  hail::(anonymous namespace)::get_module_dir()+0xb
#
# Core dump written. Default location: /share/core or core.4007
#
# An error report file with more information is saved as:
# /share/hs_err_pid4007.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
ERROR:root:Exception while sending command.
Traceback (most recent call last):
  File "/home/vagrant/miniconda3/envs/hail/lib/python3.6/site-packages/py4j/java_gateway.py", line 1159, in send_command
    raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vagrant/miniconda3/envs/hail/lib/python3.6/site-packages/py4j/java_gateway.py", line 985, in send_command
    response = connection.send_command(command)
  File "/home/vagrant/miniconda3/envs/hail/lib/python3.6/site-packages/py4j/java_gateway.py", line 1164, in send_command
    "Error while receiving", e, proto.ERROR_ON_RECEIVE)
py4j.protocol.Py4JNetworkError: Error while receiving
---------------------------------------------------------------------------
Py4JError                                 Traceback (most recent call last)
<ipython-input-2-e9881fe2c7b5> in <module>
----> 1 mt = hl.balding_nichols_model(n_populations=3, n_samples=50, n_variants=100).count()

~/miniconda3/envs/hail/lib/python3.6/site-packages/hail/matrixtable.py in count(self)
   2419             Number of rows, number of cols.
   2420         """
-> 2421         return (self.count_rows(), self.count_cols())
   2422 
   2423     @typecheck_method(output=str,

</home/vagrant/miniconda3/envs/hail/lib/python3.6/site-packages/decorator.py:decorator-gen-1104> in count_rows(self, _localize)

~/miniconda3/envs/hail/lib/python3.6/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
    583     def wrapper(__original_func, *args, **kwargs):
    584         args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 585         return __original_func(*args_, **kwargs_)
    586 
    587     return wrapper

~/miniconda3/envs/hail/lib/python3.6/site-packages/hail/matrixtable.py in count_rows(self, _localize)
   2374         ir = TableCount(MatrixRowsTable(self._mir))
   2375         if _localize:
-> 2376             return Env.backend().execute(ir)
   2377         else:
   2378             return construct_expr(ir, hl.tint64)

~/miniconda3/envs/hail/lib/python3.6/site-packages/hail/backend/backend.py in execute(self, ir, timed)
    106 
    107     def execute(self, ir, timed=False):
--> 108         result = json.loads(Env.hc()._jhc.backend().executeJSON(self._to_java_ir(ir)))
    109         value = ir.typ._from_json(result['value'])
    110         timings = result['timings']

~/miniconda3/envs/hail/lib/python3.6/site-packages/py4j/java_gateway.py in __call__(self, *args)
   1255         answer = self.gateway_client.send_command(command)
   1256         return_value = get_return_value(
-> 1257             answer, self.gateway_client, self.target_id, self.name)
   1258 
   1259         for temp_arg in temp_args:

~/miniconda3/envs/hail/lib/python3.6/site-packages/hail/utils/java.py in deco(*args, **kwargs)
    226         import pyspark
    227         try:
--> 228             return f(*args, **kwargs)
    229         except py4j.protocol.Py4JJavaError as e:
    230             s = e.java_exception.toString()

~/miniconda3/envs/hail/lib/python3.6/site-packages/py4j/protocol.py in get_return_value(answer, gateway_client, target_id, name)
    334             raise Py4JError(
    335                 "An error occurred while calling {0}{1}{2}".
--> 336                 format(target_id, ".", name))
    337     else:
    338         type = answer[1]

Py4JError: An error occurred while calling o50.executeJSON

Doubts:

Thanks!
Pau

2

what version of GLIBC do you have?
http://ask.xmodulo.com/check-glibc-version-linux.html

3

Hi tpoterba,
ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23
Copyright © 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

thanks

4

SIGILL makes me think this is an architecture issue. We build for what I think is a rather generic x86 CPU but it looks like your CPU is missing some feature. I’d try building from source using the developer docs.

5

I’m curious what CPU you’re running too.

6

Hi,
here is the output of lscpu

lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel® Xeon® CPU E5-1650 v3 @ 3.50GHz
Stepping: 2
CPU MHz: 3497.074
BogoMIPS: 6994.14
Hypervisor vendor: Oracle
Virtualization type: full
L1d cache: 32K
L1d cache: 32K
L2d cache: 6144K
NUMA node0 CPU(s): 0,1
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl cpuid pni ssse3 lahf_lm pti

I will try your build suggestion.

7

It appears your CPU does not have the AVX flag. We compile for architectures that support that flag. It’s possible the memcpy was implemented using one of these wide registers. I’m pretty confident building from source will resolve your issue. My apologies that you ran into this!

8

It appears that the AVX flag issue is related to the Virtualbox version I’m using
https://forums.virtualbox.org/viewtopic.php?f=15&t=67731
Thanks for pointing me in that direction!

9

If you can use docker, this is a pretty simple one https://hub.docker.com/r/danking00/hail/tags

FROM python:3.6
RUN apt-get update && apt-get install -y \
    openjdk-8-jre \
    && rm -rf /var/lib/apt/lists/* && \
    pip3 --no-cache-dir install hail ipython
10

Thanks, but I can’t use docker in my host institute