The Problem
I’m trying to get HAIL to work on the Community Edition of Databricks. I’ve tried everything. Is there any clear way of installing Hail on Databricks and making it work ?
there’s some docs on that here: https://docs.databricks.com/applications/genomics/hail.html
How do i find the latest links (assuming python 3) for the two below ?
“First download Hail’s Python and Java libraries to your computer:
https://storage.googleapis.com/hail-common/hail-tutorial-databricks.jar
https://storage.googleapis.com/hail-common/hail-devel-py2.7-databricks.egg”
I dont want older links, i need the updated ones…
Where did you see that? I know it was once required to upload a jar/egg manually to use Hail on Databricks, but the ENABLE_HAIL=1 flag should now do that automatically. I don’t know what version is running, though.
I tried the ‘ENABLE_HAIL=1’ flag, never got Hail to work, when you try to import hail as hl it just errors out and says it can’t find the library. I tried the latest Databricks cluster/notebook, doesn’t work.
I tried on the Community Edition and the free 14-day trial. I even tried PyPI install of the hail library, it just froze up.
Im just trying to find out how to run i guess Hail on the 14-day trial edition if at all possible…
You’ll need to talk to Databricks about this, we have no control over how they install hail.
If you want to experiment with hail on small data, try pip installing it on a Mac or Linux laptop or server.
If you want to use it to process a lot of data, we support Google Cloud Platform pretty well with hailctl dataproc (installed when you pip install hail). 100 core clusters on dataproc are in the ballpark of dollars an hour. 100 cores is quite a lot of compute for most jobs!
Is there a particular use-case you have in mind?
Got Google Cloud Platform to install it the first time, thanks man ! ! ! !
I gave up on Databricks, I’ve tried everything to make it work on the community and 14-day free trial, it seemed to freeze up on Databricks (so bad that the notebook couldn’t even accept a python ‘comment’ command, it was nuts). Thanks again man.