Hail 0.2 class not found exception on EMR

It’s totally possible to compile your own! I haven’t done it in a while (since making that .jar and .zip) so I could be wrong about specifics, but all you need to do is pass versions for Spark, Breeze, and py4j:

./gradlew -Dspark.version=2.3.0 -Dbreeze.version=0.13.2 -Dpy4j.version=0.10.6 shadowJar archiveZip

I just looked up the breeze / py4j versions for spark 2.3.0 so these should be correct.

1 Like