# Hail tutorials work, but otherwise hail does not import

**URL:** <https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392>\
**Category:** Help \[0.1\]\
**Created:** [January 5, 2018, 8:09pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392 "2018-01-05T20:09:33Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:09pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/1 "2018-01-05T20:09:33Z")

</div>

Hi,  
I am having problems importing the hail package into python.

Weirdly, I can run the jupyter hail tutorials no problem.

However, creating a new jupyter notebook in that same tutorials folder and trying to import hail there produces the error  
ImportError: No module named hail

Running interactive python from elsewhere and attempting to import hail produces the same No module named hail error.

I have set SPARK\_HOME and PATH variables as in the ‘Getting started’ instructions, and they now contain the locations of the relevant packages.

I generally use anaconda environments for running python, is there a way to easily make hail available within an anaconda environment that I have created?

Help is appreciated, many thanks!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:12pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/2 "2018-01-05T20:12:27Z")

</div>

Since we haven’t registered Hail on PyPI, I think it’s hard to make a conda yaml file to set up everything automatically.

The problem is probably that the Hail python zip isn’t on your path – are you using the `jhail` script in the distribution? That sets it up. Otherwise, you’ll need to add the Hail python library to the PYTHONPATH environment variable.

Let me know if this is confusing (I’ve slightly confused myself).

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:17pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/3 "2018-01-05T20:17:26Z")

</div>

Yes, I was using the jhail command, I had forgotten that.

What exactly should I add to my PYTHONPATH variable? This variable is indeed empty right now, as it’s discouraged if you are using anaconda. However I am not afraid to change it.

Thank you for the almost instantaneous response!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:20pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/4 "2018-01-05T20:20:14Z")

</div>

Hmm, interesting – I should really look into what Anaconda prefers, it’s a great Python installation and we should totally play nicely with it. I use Anaconda and PYTHONPATH on my local machine and everything seems to work (I have Hail and py4j in my PYTHONPATH and they persist between virtual envs).

Here’s what mine looks like:

```auto
export PYTHONPATH="$PYTHONPATH:$HAIL_HOME/python:$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.10.3-src.zip"

```

It sounds like you have the Python .zip, so the $HAIL\_HOME/python should be the path to the .zip there.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:20pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/5 "2018-01-05T20:20:37Z")

</div>

also if you have a different Spark, the py4j version might be different.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:27pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/6 "2018-01-05T20:27:49Z")

</div>

I really like Anaconda. They say it’s ok to mix it with having a filled PYTHONPATH variable, but just sometimes causes problems depending on what’s being done.

So I have HAIL\_HOME and SPARK\_HOME variables set already, according to the getting started instructions.

Are you saying those might need to be altered, or do you think I can copy what you have except for the ‘python/lib/py4j-0.10.3-src.zip’ part? (Or maybe even that will be the same for me, I just have to look and see what seems to correspond on my system).

I should have the ‘correct’ spark; I hadn’t used spark before so I just downloaded the one that should ‘match’ with hail.

Thanks again!

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:31pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/7 "2018-01-05T20:31:11Z")

</div>

Oh fun.

I copied what you had exactly, and I got a new error:

> Traceback (most recent call last):  
> File “”, line 1, in   
> File “/Applications/hail/python/hail/ **init**.py”, line 1, in   
> import hail.expr  
> File “/Applications/hail/python/hail/expr.py”, line 2, in   
> from hail.java import scala\_object, Env, jset  
> File “/Applications/hail/python/hail/java.py”, line 7, in   
> from decorator import decorator  
> ImportError: No module named decorator

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:31pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/8 "2018-01-05T20:31:48Z")

</div>

Oh, good – just pip install decorator.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:33pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/9 "2018-01-05T20:33:24Z")

</div>

(I thought decorator was bundled with Anaconda… I guess not)

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:34pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/10 "2018-01-05T20:34:14Z")

</div>

I see Decorator 4.1.2 seems to be available through Anaconda. Is a specific version required? Thanks again.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:34pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/11 "2018-01-05T20:34:24Z")

</div>

I think anything will work.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:40pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/12 "2018-01-05T20:40:25Z")

</div>

OK, got decorator.

This now doesn’t throw an error:

> from hail import \*

But this does:

> hc = HailContext()

> Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties  
> Setting default log level to “WARN”.  
> To adjust logging level use sc.setLogLevel(newLevel).  
> Traceback (most recent call last):  
> File “”, line 1, in   
> File “”, line 2, in **init**  
> File “/Applications/hail/python/hail/typecheck/check.py”, line 245, in \_typecheck  
> return f(\*args, \*\*kwargs)  
> File “/Applications/hail/python/hail/context.py”, line 88, in **init**  
> parquet\_compression, min\_block\_size, branching\_factor, tmp\_dir)  
> TypeError: ‘JavaPackage’ object is not callable

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:42pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/13 "2018-01-05T20:42:55Z")

</div>

OK, that probably means that the jar isn’t accessible on your path. you’ll need to set SPARK\_CLASSPATH.

I just found the script used in jhail/ihail/etc. It sets these:

```auto
export HAIL_HOME="$(dirname "$(cd "$(dirname $0)" && pwd)")"
export PYTHONPATH="$HAIL_HOME/python:$SPARK_HOME/python:$(echo ${SPARK_HOME}/python/lib/py4j-*-src.zip | tr '\n' ':')$PYTHONPATH"
export SPARK_CLASSPATH=$HAIL_HOME/jars/hail-all-spark.jar

```

Since you’ve got the other two working, just copy in the SPARK\_CLASSPATH one.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:45pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/14 "2018-01-05T20:45:54Z")

</div>

Yeah it worked:

> Welcome to  
> \_\_ \_\_ \<\>\_\_  
> / /_/ /\_\_ \_\_/ /  
> / \_\_ / \_ `/ / /  
> /_/ /_/\_,_/_/_/ version 0.1-b374ef3

Thanks very much!!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:46pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/15 "2018-01-05T20:46:17Z")

</div>

great! Good to know what got you stuck, we’ll try to clarify the docs / getting started page in the future.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 8:49pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/16 "2018-01-05T20:49:20Z")

</div>

It just comes down to those pesky environment variables. Perhaps you could include something about these in a troubleshooting section or something? Also a note that people should not assume that because the tutorials run, everything is set?  
I’m looking forward to messing with some vds’s now. Thanks again.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [January 5, 2018, 8:49pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/17 "2018-01-05T20:49:52Z")

</div>

> [@rmccole](#):
>
> Perhaps you could include something about these in a troubleshooting section or something?

Yeah, that’s a good idea.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 5, 2018, 9:00pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/18 "2018-01-05T21:00:19Z")

</div>

Just to add I think Decorator does come with Anaconda (I had an old version), but even so I needed to add it to the Anaconda environment I was working in for it to be available.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [January 10, 2018, 4:43pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/19 "2018-01-10T16:43:27Z")

</div>

How did you get Hail?

The files in the `scripts` folder are not meant to be used with the source (from git / GitHub) distribution of hail. Those are files that are used by the packaging mechanism to produce the pre-packaged Hail distributions.

If you downloaded the pre-built Hail distribution from the [Getting Started page](https://hail.is/docs/stable/getting_started.html) and you encountered these problems, then we should understand why and fix the pre-built distribution.

I strongly recommend using the pre-built distribution mentioned on the getting started page.

---

<div class="post-metadata">

**Author:** ![rmccole](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rmccole](https://discuss.hail.is/u/rmccole)\
**Post date:** [January 12, 2018, 3:47pm UTC](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392/20 "2018-01-12T15:47:39Z")

</div>

I did get it from the Getting Started page, not from GitHub.

[Next page](https://discuss.hail.is/t/hail-tutorials-work-but-otherwise-hail-does-not-import/392.md?page=2)
