Hey everyone, I’ve been having trouble getting Hail installed and the tests to run correctly. Everything seems to work fine until I run the tests. It passes most tests, but fails out with this error,
14:17:27.809 [ERROR] [system.err] hail: info: while writing:
14:17:27.809 [ERROR] [system.err] /tmp/testExportKT.tsv
14:17:27.810 [ERROR] [system.err] merge time: 7.677ms
14:17:28.591 [ERROR] [system.err] hail: info: Coerced sorted dataset
14:17:30.368 [ERROR] [system.err] .hail: info: Coerced sorted dataset
14:17:31.306 [ERROR] [system.err] ...
14:17:31.904 [ERROR] [system.err] ==================================================================
14:17:31.905 [ERROR] [system.err] ERROR: test_dataset (hail.tests.ContextTests)
14:17:31.905 [ERROR] [system.err] ----------------------------------------------------------------------
14:17:31.905 [ERROR] [system.err] Traceback (most recent call last):
14:17:31.905 [ERROR] [system.err] File "/scratch/PI/dpwall/computeEnvironments/hail/python/hail/tests.py", line 181, in test_dataset
14:17:31.906 [ERROR] [system.err] sample2.grm('gcta-grm-bin', '/tmp/sample2.grm')
14:17:31.906 [ERROR] [system.err] File "/scratch/PI/dpwall/computeEnvironments/hail/python/hail/dataset.py", line 1988, in grm
14:17:31.906 [ERROR] [system.err] self.hc._run_command(self, pargs)
14:17:31.906 [ERROR] [system.err] File "/scratch/PI/dpwall/computeEnvironments/hail/python/hail/context.py", line 90, in _run_command
14:17:31.907 [ERROR] [system.err] raise_py4j_exception(e)
14:17:31.907 [ERROR] [system.err] File "/scratch/PI/dpwall/computeEnvironments/hail/python/hail/java.py", line 87, in raise_py4j_exception
14:17:31.907 [ERROR] [system.err] raise FatalError(msg, e.java_exception)
14:17:31.908 [ERROR] [system.err] FatalError: NoSuchMethodError: breeze.linalg.DenseVector$.canSetD()Lbreeze/generic/UFunc$InPlaceImpl2;
14:17:31.908 [ERROR] [system.err]
14:17:31.908 [ERROR] [system.err] ----------------------------------------------------------------------
14:17:31.908 [ERROR] [system.err] Ran 7 tests in 83.523s
It seems like it fails out due to missing a certain breeze function in scala. Breeze seems to have been installed at initial compile time, and I am not sure how to solve this problem. I should also note that I have tried testing the latest version of hail with spark 2.1.0 on both my local macbook and our shared HPC cluster (with Spark) and it fails out at the exact same place. However, when I switch to spark 2.0.2 it is able to complete almost all the tests on both my laptop and cluster. Perhaps a problem with Spark 2.1.0 paths? Any suggestion would be really appreciated. Thank you!