I have booted a spark cluster in the cloud using Elasticluster. Elasticluster uses the Apache Bigtop distribution, which currently builds with Spark 2.1.1.
I have tried to build a hail jar in this cluster following the official hail docs but when I execute ./gradlew -Dspark.version=2.1.1 shadowJar I get this error:
ubuntu@master001:~/hail$ ./gradlew -Dspark.version=2.1.1 shadowJar
Downloading https://services.gradle.org/distributions/gradle-2.14.1-bin.zip
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Unzipping /home/ubuntu/.gradle/wrapper/dists/gradle-2.14.1-bin/2r579t5wehc7ew5kc8vfqezww/gradle-2.14.1-bin.zip to /home/ubuntu/.gradle/wrapper/dists/gradle-2.14.1-bin/2r579t5wehc7ew5kc8vfqezww
Set executable permissions for: /home/ubuntu/.gradle/wrapper/dists/gradle-2.14.1-bin/2r579t5wehc7ew5kc8vfqezww/gradle-2.14.1/bin/gradle
Download https://plugins.gradle.org/m2/com/gradle/build-scan-plugin/1.0/build-scan-plugin-1.0.pom
Download https://jcenter.bintray.com/com/github/jengelman/gradle/plugins/shadow/1.2.3/shadow-1.2.3.pom
Download https://repo1.maven.org/maven2/de/undercouch/gradle-download-task/3.2.0/gradle-download-task-3.2.0.pom
Download https://repo1.maven.org/maven2/org/jdom/jdom2/2.0.5/jdom2-2.0.5.pom
Download https://repo1.maven.org/maven2/org/ow2/asm/asm/5.0.3/asm-5.0.3.pom
Download https://repo1.maven.org/maven2/org/ow2/asm/asm-parent/5.0.3/asm-parent-5.0.3.pom
Download https://repo1.maven.org/maven2/org/ow2/ow2/1.3/ow2-1.3.pom
Download https://repo1.maven.org/maven2/org/ow2/asm/asm-commons/5.0.3/asm-commons-5.0.3.pom
Download https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom
Download https://repo1.maven.org/maven2/org/apache/apache/9/apache-9.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.4/ant-1.9.4.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant-parent/1.9.4/ant-parent-1.9.4.pom
Download https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/2.0.6/plexus-utils-2.0.6.pom
Download https://repo1.maven.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom
Download https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-backports-compat23/2.4.4/groovy-backports-compat23-2.4.4.pom
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.pom
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.2/httpcomponents-client-4.5.2.pom
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/project/7/project-7.pom
Download https://repo1.maven.org/maven2/org/apache/apache/13/apache-13.pom
Download https://repo1.maven.org/maven2/org/ow2/asm/asm-tree/5.0.3/asm-tree-5.0.3.pom
Download https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.4/ant-launcher-1.9.4.pom
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.pom
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.4/httpcomponents-core-4.4.4.pom
Download https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom
Download https://repo1.maven.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom
Download https://repo1.maven.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom
Download https://plugins.gradle.org/m2/com/gradle/build-scan-plugin/1.0/build-scan-plugin-1.0.jar
Download https://jcenter.bintray.com/com/github/jengelman/gradle/plugins/shadow/1.2.3/shadow-1.2.3.jar
Download https://repo1.maven.org/maven2/de/undercouch/gradle-download-task/3.2.0/gradle-download-task-3.2.0.jar
Download https://repo1.maven.org/maven2/org/jdom/jdom2/2.0.5/jdom2-2.0.5.jar
Download https://repo1.maven.org/maven2/org/ow2/asm/asm/5.0.3/asm-5.0.3.jar
Download https://repo1.maven.org/maven2/org/ow2/asm/asm-commons/5.0.3/asm-commons-5.0.3.jar
Download https://repo1.maven.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar
Download https://repo1.maven.org/maven2/org/apache/ant/ant/1.9.4/ant-1.9.4.jar
Download https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/2.0.6/plexus-utils-2.0.6.jar
Download https://repo1.maven.org/maven2/org/codehaus/groovy/groovy-backports-compat23/2.4.4/groovy-backports-compat23-2.4.4.jar
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar
Download https://repo1.maven.org/maven2/org/ow2/asm/asm-tree/5.0.3/asm-tree-5.0.3.jar
Download https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.9.4/ant-launcher-1.9.4.jar
Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.4.4/httpcore-4.4.4.jar
Download https://repo1.maven.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar
Download https://repo1.maven.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
3da0e7424af0
FAILURE: Build failed with an exception.
* Where:
Build file '/home/ubuntu/hail/build.gradle' line: 57
* What went wrong:
A problem occurred evaluating root project 'hail'.
> Unknown Spark version 2.1.1. Set breeze.version and py4j.version properties for Spark 2.1.1.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Is it possible to build hail with Spark 2.1.1?
Thanks in advance for any help.