Hello,
Our security team requires us to use containers with a minimal OS. As such, the ps command installed in our base image cannot be used with the -p flag. When I try to read a matrix, Hail starts initialising, and fails with the error
Initializing Hail with default parameters...
ps: invalid option -- 'p'
Is there a known solution to this problem?
Thanks
What is the stack trace of the error that you’re seeing?
This is likely within a dependency, but I’d like to figure out which one, so we may be able to come up with a workaround that can fit your use case.
Hi, thanks for the quick response!
this is what I get
>>> import hail as hl
>>> mt = hl.balding_nichols_model(3, 10000, 100)
Initializing Hail with default parameters...
ps: invalid option -- 'p'
BusyBox v1.36.1 (2024-03-27 13:51:06 UTC) multi-call binary.
Usage: ps [-o COL1,COL2=HEADER] [-T]
Show list of processes
-o COL1,COL2=HEADER Select columns for display
-T Show threads
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007ffffe9a7522, pid=341, tid=0x00007ffffcb3b6c0
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.52.0.23-CA-linux64) (8.0_282-b08) (build 1.8.0_282-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.282-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V [libjvm.so+0x9a7522] StealTask::do_it(GCTaskManager*, unsigned int)+0x112
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# //hs_err_pid341.log
#
# If you would like to submit a bug report, please visit:
# http://www.azulsystems.com/support/
#
ERROR:root:Exception while sending command.
Traceback (most recent call last):
File "/root/conda/lib/python3.12/site-packages/py4j/clientserver.py", line 516, in send_command
raise Py4JNetworkError("Answer from Java side is empty")
py4j.protocol.Py4JNetworkError: Answer from Java side is empty
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/conda/lib/python3.12/site-packages/py4j/java_gateway.py", line 1038, in send_command
response = connection.send_command(command)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/py4j/clientserver.py", line 539, in send_command
raise Py4JNetworkError(
py4j.protocol.Py4JNetworkError: Error while sending or receiving
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/root/conda/lib/python3.12/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 583, in wrapper
args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 548, in check_all
args_.append(arg_check(args[i], name, arg_name, checker))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 591, in arg_check
return checker.check(arg, function_name, arg_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 81, in check
return tc.check(x, caller, param)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 328, in check
return f(tc.check(x, caller, param))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/genetics/reference_genome.py", line 10, in <lambda>
reference_genome_type = oneof(transformed((str, lambda x: hl.get_reference(x))), rg_type)
^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/context.py", line 908, in get_reference
Env.hc()
File "/root/conda/lib/python3.12/site-packages/hail/utils/java.py", line 64, in hc
init()
File "/root/conda/lib/python3.12/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 584, in wrapper
return __original_func(*args_, **kwargs_)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/context.py", line 386, in init
return init_spark(
^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), **kw)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/typecheck/check.py", line 584, in wrapper
return __original_func(*args_, **kwargs_)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/context.py", line 476, in init_spark
backend = SparkBackend(
^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/hail/backend/spark_backend.py", line 141, in __init__
jhc = hail_package.HailContext.apply(jbackend, branching_factor, optimizer_iterations)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/py4j/java_gateway.py", line 1321, in __call__
return_value = get_return_value(
^^^^^^^^^^^^^^^^^
File "/root/conda/lib/python3.12/site-packages/py4j/protocol.py", line 334, in get_return_value
raise Py4JError(
py4j.protocol.Py4JError: An error occurred while calling z:is.hail.HailContext.apply
>>>