# Hail Tutorial Error

**URL:** <https://discuss.hail.is/t/hail-tutorial-error/3792>\
**Category:** Hail Query & hailctl\
**Created:** [March 31, 2024, 7:35pm UTC](https://discuss.hail.is/t/hail-tutorial-error/3792 "2024-03-31T19:35:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![andie\_d](https://avatars.discourse-cdn.com/v4/letter/a/41988e/32.png) [@andie\_d](https://discuss.hail.is/u/andie_d)\
**Post date:** [March 31, 2024, 7:35pm UTC](https://discuss.hail.is/t/hail-tutorial-error/3792/1 "2024-03-31T19:35:35Z")

</div>

I am trying to follow the general hail tutorial using my Mac OS (M2 Chip). I followed the download instructions and downloaded Java 8 with Azul. I continue to get the follow error when I simply try to initialize Hail: hl.init()

Error:  
Py4JJavaError Traceback (most recent call last)  
Cell In[3], line 14  
11 from pprint import pprint  
13 # Initialize Hail  
—\> 14 hl.init()

File :2, in init(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, backend, driver\_cores, driver\_memory, worker\_cores, worker\_memory, gcs\_requester\_pays\_configuration, regions, gcs\_bucket\_allow\_list)

File ~/mambaforge/lib/python3.10/site-packages/hail/typecheck/check.py:584, in \_make\_dec..wrapper(\_\_original\_func, \*args, \*\*kwargs)  
581 @decorator  
582 def wrapper(_original\_func: Callable[…, T], \*args, \*\*kwargs) → T:  
583 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method)  
 → 584 return _original\_func(\*args, \*\*kwargs_)

File ~/mambaforge/lib/python3.10/site-packages/hail/context.py:386, in init(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, backend, driver\_cores, driver\_memory, worker\_cores, worker\_memory, gcs\_requester\_pays\_configuration, regions, gcs\_bucket\_allow\_list)  
366 return hail\_event\_loop().run\_until\_complete(  
367 init\_batch(  
368 log=log,  
(…)  
383 )  
384 )  
385 if backend == ‘spark’:  
 → 386 return init\_spark(  
387 sc=sc,  
388 app\_name=app\_name,  
389 master=master,  
390 local=local,  
391 min\_block\_size=min\_block\_size,  
392 branching\_factor=branching\_factor,  
393 spark\_conf=spark\_conf,  
394 \_optimizer\_iterations=\_optimizer\_iterations,  
395 log=log,  
396 quiet=quiet,  
397 append=append,  
398 tmp\_dir=tmp\_dir,  
399 local\_tmpdir=local\_tmpdir,  
400 default\_reference=default\_reference,  
401 global\_seed=global\_seed,  
402 skip\_logging\_configuration=skip\_logging\_configuration,  
403 gcs\_requester\_pays\_configuration=gcs\_requester\_pays\_configuration,  
404 )  
405 if backend == ‘local’:  
406 return init\_local(  
407 log=log,  
408 quiet=quiet,  
(…)  
414 gcs\_requester\_pays\_configuration=gcs\_requester\_pays\_configuration,  
415 )

File :2, in init\_spark(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, gcs\_requester\_pays\_configuration)

File ~/mambaforge/lib/python3.10/site-packages/hail/typecheck/check.py:584, in \_make\_dec..wrapper(\_\_original\_func, \*args, \*\*kwargs)  
581 @decorator  
582 def wrapper(_original\_func: Callable[…, T], \*args, \*\*kwargs) → T:  
583 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method)  
 → 584 return _original\_func(\*args, \*\*kwargs_)

File ~/mambaforge/lib/python3.10/site-packages/hail/context.py:476, in init\_spark(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, gcs\_requester\_pays\_configuration)  
467 app\_name = app\_name or ‘Hail’  
468 (  
469 gcs\_requester\_pays\_project,  
470 gcs\_requester\_pays\_buckets,  
(…)  
474 )  
475 )  
 → 476 backend = SparkBackend(  
477 idempotent,  
478 sc,  
479 spark\_conf,  
480 app\_name,  
481 master,  
482 local,  
483 log,  
484 quiet,  
485 append,  
486 min\_block\_size,  
487 branching\_factor,  
488 tmpdir,  
489 local\_tmpdir,  
490 skip\_logging\_configuration,  
491 optimizer\_iterations,  
492 gcs\_requester\_pays\_project=gcs\_requester\_pays\_project,  
493 gcs\_requester\_pays\_buckets=gcs\_requester\_pays\_buckets,  
494 )  
495 if not backend.fs.exists(tmpdir):  
496 backend.fs.mkdir(tmpdir)

File ~/mambaforge/lib/python3.10/site-packages/hail/backend/spark\_backend.py:126, in SparkBackend. **init** (self, idempotent, sc, spark\_conf, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmpdir, local\_tmpdir, skip\_logging\_configuration, optimizer\_iterations, gcs\_requester\_pays\_project, gcs\_requester\_pays\_buckets)  
124 jhc = hail\_package.HailContext.getOrCreate(jbackend, branching\_factor, optimizer\_iterations)  
125 else:  
 → 126 jbackend = hail\_package.backend.spark.SparkBackend.apply(  
127 jsc,  
128 app\_name,  
129 master,  
130 local,  
131 log,  
132 True,  
133 append,  
134 skip\_logging\_configuration,  
135 min\_block\_size,  
136 tmpdir,  
137 local\_tmpdir,  
138 gcs\_requester\_pays\_project,  
139 gcs\_requester\_pays\_buckets,  
140 )  
141 jhc = hail\_package.HailContext.apply(jbackend, branching\_factor, optimizer\_iterations)  
143 self.\_jsc = jbackend.sc()

File ~/mambaforge/lib/python3.10/site-packages/py4j/java\_gateway.py:1321, in JavaMember. **call** (self, \*args)  
1315 command = proto.CALL\_COMMAND\_NAME +  
1316 self.command\_header +  
1317 args\_command +  
1318 proto.END\_COMMAND\_PART  
1320 answer = self.gateway\_client.send\_command(command)  
 → 1321 return\_value = get\_return\_value(  
1322 answer, self.gateway\_client, self.target\_id, self.name)  
1324 for temp\_arg in temp\_args:  
1325 temp\_arg.\_detach()

File ~/mambaforge/lib/python3.10/site-packages/py4j/protocol.py:326, in get\_return\_value(answer, gateway\_client, target\_id, name)  
324 value = OUTPUT\_CONVERTER[type](answer[2:], gateway\_client)  
325 if answer[1] == REFERENCE\_TYPE:  
 → 326 raise Py4JJavaError(  
327 “An error occurred while calling {0}{1}{2}.\n”.  
328 format(target\_id, “.”, name), value)  
329 else:  
330 raise Py4JError(  
331 “An error occurred while calling {0}{1}{2}. Trace:\n{3}\n”.  
332 format(target\_id, “.”, name, value))

Py4JJavaError: An error occurred while calling z:is.hail.backend.spark.SparkBackend.apply.  
: java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.unsafe.array.ByteArrayMethods  
at org.apache.spark.memory.MemoryManager.defaultPageSizeBytes$lzycompute(MemoryManager.scala:264)  
at org.apache.spark.memory.MemoryManager.defaultPageSizeBytes(MemoryManager.scala:254)  
at org.apache.spark.memory.MemoryManager.$anonfun$pageSizeBytes$1(MemoryManager.scala:273)  
at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)  
at scala.Option.getOrElse(Option.scala:189)  
at org.apache.spark.memory.MemoryManager.(MemoryManager.scala:273)  
at org.apache.spark.memory.UnifiedMemoryManager.(UnifiedMemoryManager.scala:58)  
at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:207)  
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:320)  
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:194)  
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:279)  
at org.apache.spark.SparkContext.(SparkContext.scala:464)  
at is.hail.backend.spark.SparkBackend$.configureAndCreateSparkContext(SparkBackend.scala:156)  
at is.hail.backend.spark.SparkBackend$.apply(SparkBackend.scala:262)  
at is.hail.backend.spark.SparkBackend.apply(SparkBackend.scala)  
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)  
at java.base/java.lang.reflect.Method.invoke(Method.java:580)  
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
at py4j.Gateway.invoke(Gateway.java:282)  
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
at py4j.commands.CallCommand.execute(CallCommand.java:79)  
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)  
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)  
at java.base/java.lang.Thread.run(Thread.java:1583)  
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.ExceptionInInitializerError [in thread “Thread-3”]  
at org.apache.spark.unsafe.array.ByteArrayMethods.(ByteArrayMethods.java:56)  
… 25 more

hl.plot.output\_notebook()  
BokehJS 3.4.0 successfully loaded.

# Download public 1000 Genomes data

## hl.utils.get\_1kg(‘data/’) Initializing Hail with default parameters…

Py4JJavaError Traceback (most recent call last)  
Cell In[5], line 2  
1 # Download public 1000 Genomes data  
----\> 2 hl.utils.get\_1kg(‘data/’)

File ~/mambaforge/lib/python3.10/site-packages/hail/utils/tutorial.py:59, in get\_1kg(output\_dir, overwrite)  
44 def get\_1kg(output\_dir, overwrite: bool = False):  
45 “”“Download subset of the `1000 Genomes <http://www.internationalgenome.org/>`\_\_  
46 dataset and sample annotations.  
47  
(…)  
57 If `True`, overwrite any existing files/directories at `output_dir`.  
58 “””  
—\> 59 fs = Env.fs()  
61 if not \_dir\_exists(fs, output\_dir):  
62 fs.mkdir(output\_dir)

File ~/mambaforge/lib/python3.10/site-packages/hail/utils/java.py:115, in Env.fs()  
113 @staticmethod  
114 def fs():  
 → 115 return Env.backend().fs

File ~/mambaforge/lib/python3.10/site-packages/hail/utils/java.py:91, in Env.backend()  
89 @staticmethod  
90 def backend() → ‘hail.backend.Backend’:  
—\> 91 return Env.hc().\_backend

File ~/mambaforge/lib/python3.10/site-packages/hail/utils/java.py:64, in Env.hc()  
61 sys.stderr.flush()  
62 from ..context import init  
—\> 64 init()  
66 assert Env.\_hc is not None  
67 return Env.\_hc

File :2, in init(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, backend, driver\_cores, driver\_memory, worker\_cores, worker\_memory, gcs\_requester\_pays\_configuration, regions, gcs\_bucket\_allow\_list)

File ~/mambaforge/lib/python3.10/site-packages/hail/typecheck/check.py:584, in \_make\_dec..wrapper(\_\_original\_func, \*args, \*\*kwargs)  
581 @decorator  
582 def wrapper(_original\_func: Callable[…, T], \*args, \*\*kwargs) → T:  
583 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method)  
 → 584 return _original\_func(\*args, \*\*kwargs_)

File ~/mambaforge/lib/python3.10/site-packages/hail/context.py:386, in init(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, backend, driver\_cores, driver\_memory, worker\_cores, worker\_memory, gcs\_requester\_pays\_configuration, regions, gcs\_bucket\_allow\_list)  
366 return hail\_event\_loop().run\_until\_complete(  
367 init\_batch(  
368 log=log,  
(…)  
383 )  
384 )  
385 if backend == ‘spark’:  
 → 386 return init\_spark(  
387 sc=sc,  
388 app\_name=app\_name,  
389 master=master,  
390 local=local,  
391 min\_block\_size=min\_block\_size,  
392 branching\_factor=branching\_factor,  
393 spark\_conf=spark\_conf,  
394 \_optimizer\_iterations=\_optimizer\_iterations,  
395 log=log,  
396 quiet=quiet,  
397 append=append,  
398 tmp\_dir=tmp\_dir,  
399 local\_tmpdir=local\_tmpdir,  
400 default\_reference=default\_reference,  
401 global\_seed=global\_seed,  
402 skip\_logging\_configuration=skip\_logging\_configuration,  
403 gcs\_requester\_pays\_configuration=gcs\_requester\_pays\_configuration,  
404 )  
405 if backend == ‘local’:  
406 return init\_local(  
407 log=log,  
408 quiet=quiet,  
(…)  
414 gcs\_requester\_pays\_configuration=gcs\_requester\_pays\_configuration,  
415 )

File :2, in init\_spark(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, gcs\_requester\_pays\_configuration)

File ~/mambaforge/lib/python3.10/site-packages/hail/typecheck/check.py:584, in \_make\_dec..wrapper(\_\_original\_func, \*args, \*\*kwargs)  
581 @decorator  
582 def wrapper(_original\_func: Callable[…, T], \*args, \*\*kwargs) → T:  
583 args, kwargs_ = check\_all(\_\_original\_func, args, kwargs, checkers, is\_method=is\_method)  
 → 584 return _original\_func(\*args, \*\*kwargs_)

File ~/mambaforge/lib/python3.10/site-packages/hail/context.py:476, in init\_spark(sc, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmp\_dir, default\_reference, idempotent, global\_seed, spark\_conf, skip\_logging\_configuration, local\_tmpdir, \_optimizer\_iterations, gcs\_requester\_pays\_configuration)  
467 app\_name = app\_name or ‘Hail’  
468 (  
469 gcs\_requester\_pays\_project,  
470 gcs\_requester\_pays\_buckets,  
(…)  
474 )  
475 )  
 → 476 backend = SparkBackend(  
477 idempotent,  
478 sc,  
479 spark\_conf,  
480 app\_name,  
481 master,  
482 local,  
483 log,  
484 quiet,  
485 append,  
486 min\_block\_size,  
487 branching\_factor,  
488 tmpdir,  
489 local\_tmpdir,  
490 skip\_logging\_configuration,  
491 optimizer\_iterations,  
492 gcs\_requester\_pays\_project=gcs\_requester\_pays\_project,  
493 gcs\_requester\_pays\_buckets=gcs\_requester\_pays\_buckets,  
494 )  
495 if not backend.fs.exists(tmpdir):  
496 backend.fs.mkdir(tmpdir)

File ~/mambaforge/lib/python3.10/site-packages/hail/backend/spark\_backend.py:126, in SparkBackend. **init** (self, idempotent, sc, spark\_conf, app\_name, master, local, log, quiet, append, min\_block\_size, branching\_factor, tmpdir, local\_tmpdir, skip\_logging\_configuration, optimizer\_iterations, gcs\_requester\_pays\_project, gcs\_requester\_pays\_buckets)  
124 jhc = hail\_package.HailContext.getOrCreate(jbackend, branching\_factor, optimizer\_iterations)  
125 else:  
 → 126 jbackend = hail\_package.backend.spark.SparkBackend.apply(  
127 jsc,  
128 app\_name,  
129 master,  
130 local,  
131 log,  
132 True,  
133 append,  
134 skip\_logging\_configuration,  
135 min\_block\_size,  
136 tmpdir,  
137 local\_tmpdir,  
138 gcs\_requester\_pays\_project,  
139 gcs\_requester\_pays\_buckets,  
140 )  
141 jhc = hail\_package.HailContext.apply(jbackend, branching\_factor, optimizer\_iterations)  
143 self.\_jsc = jbackend.sc()

File ~/mambaforge/lib/python3.10/site-packages/py4j/java\_gateway.py:1321, in JavaMember. **call** (self, \*args)  
1315 command = proto.CALL\_COMMAND\_NAME +  
1316 self.command\_header +  
1317 args\_command +  
1318 proto.END\_COMMAND\_PART  
1320 answer = self.gateway\_client.send\_command(command)  
 → 1321 return\_value = get\_return\_value(  
1322 answer, self.gateway\_client, self.target\_id, self.name)  
1324 for temp\_arg in temp\_args:  
1325 temp\_arg.\_detach()

File ~/mambaforge/lib/python3.10/site-packages/py4j/protocol.py:326, in get\_return\_value(answer, gateway\_client, target\_id, name)  
324 value = OUTPUT\_CONVERTER[type](answer[2:], gateway\_client)  
325 if answer[1] == REFERENCE\_TYPE:  
 → 326 raise Py4JJavaError(  
327 “An error occurred while calling {0}{1}{2}.\n”.  
328 format(target\_id, “.”, name), value)  
329 else:  
330 raise Py4JError(  
331 “An error occurred while calling {0}{1}{2}. Trace:\n{3}\n”.  
332 format(target\_id, “.”, name, value))

Py4JJavaError: An error occurred while calling z:is.hail.backend.spark.SparkBackend.apply.  
: java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.unsafe.array.ByteArrayMethods  
at org.apache.spark.memory.MemoryManager.defaultPageSizeBytes$lzycompute(MemoryManager.scala:264)  
at org.apache.spark.memory.MemoryManager.defaultPageSizeBytes(MemoryManager.scala:254)  
at org.apache.spark.memory.MemoryManager.$anonfun$pageSizeBytes$1(MemoryManager.scala:273)  
at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.java:23)  
at scala.Option.getOrElse(Option.scala:189)  
at org.apache.spark.memory.MemoryManager.(MemoryManager.scala:273)  
at org.apache.spark.memory.UnifiedMemoryManager.(UnifiedMemoryManager.scala:58)  
at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:207)  
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:320)  
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:194)  
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:279)  
at org.apache.spark.SparkContext.(SparkContext.scala:464)  
at is.hail.backend.spark.SparkBackend$.configureAndCreateSparkContext(SparkBackend.scala:156)  
at is.hail.backend.spark.SparkBackend$.apply(SparkBackend.scala:262)  
at is.hail.backend.spark.SparkBackend.apply(SparkBackend.scala)  
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)  
at java.base/java.lang.reflect.Method.invoke(Method.java:580)  
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
at py4j.Gateway.invoke(Gateway.java:282)  
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
at py4j.commands.CallCommand.execute(CallCommand.java:79)  
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)  
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)  
at java.base/java.lang.Thread.run(Thread.java:1583)  
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.ExceptionInInitializerError [in thread “Thread-3”]  
at org.apache.spark.unsafe.array.ByteArrayMethods.(ByteArrayMethods.java:56)  
… 25 more