# HailContext tmp\_dir

**URL:** <https://discuss.hail.is/t/hailcontext-tmp-dir/323>\
**Category:** Help \[0.1\]\
**Created:** [October 16, 2017, 11:04pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323 "2017-10-16T23:04:08Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rca](https://avatars.discourse-cdn.com/v4/letter/r/9fc29f/32.png) [@rca](https://discuss.hail.is/u/rca)\
**Post date:** [October 16, 2017, 11:04pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/1 "2017-10-16T23:04:08Z")

</div>

Hi,

I’ve noticed that the documentation for HailContext indicates tmp\_dir defaults to /tmp. Is there a way to change this default through a command line parameter or environment variable (it doesn’t seem to recognize TMP, TMPDIR or TMP\_DIR)?

Thanks,  
-rca

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 18, 2017, 4:09pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/2 "2017-10-18T16:09:36Z")

</div>

The `HailContext` constructor does not consult any environment variables when choosing the temporary directory.

The subtext here seems to be that you cannot modify your script like this:

```auto
import os

hc = HailContext(tmp_dir=os.environ['TMPDIR'] if 'TMPDIR' in os.environ else '/tmp')

```

Under what circumstances are you running hail?

---

<div class="post-metadata">

**Author:** ![rca](https://avatars.discourse-cdn.com/v4/letter/r/9fc29f/32.png) [@rca](https://discuss.hail.is/u/rca)\
**Post date:** [October 18, 2017, 8:00pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/3 "2017-10-18T20:00:33Z")

</div>

Hi Dan,

I’ve been running hail on a UGE cluster very similar to the Broad’s own cluster, but configured for /scratch/$USER to be the location for temporary files and very little space allocated to /tmp. While debugging some code by using an interactive job and running python/hail from a shell I tried exporting a VDS to VCF and ran into a problem running out of space on /tmp (despite setting /scratch/$USER for the previously mentioned environment variables, SPARK\_LOCAL\_DIRS, and with \_JAVA\_OPTIONS=-Djava.io.tmpdir).

I found that using a hardcoded path in the constructor was effective (i.e. hc = HailContext(tmp\_dir="/scratch/rca"), but this is obviously not ideal.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 19, 2017, 12:23am UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/4 "2017-10-19T00:23:38Z")

</div>

Is the reason it is not ideal because the setting is per-script rather than per shell / environment / machine?

---

<div class="post-metadata">

**Author:** ![rca](https://avatars.discourse-cdn.com/v4/letter/r/9fc29f/32.png) [@rca](https://discuss.hail.is/u/rca)\
**Post date:** [October 19, 2017, 3:01pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/5 "2017-10-19T15:01:19Z")

</div>

Exactly. Since the entire cluster is configured for /scratch to be used as a temporary directory, and /tmp to have limited space, it would be ideal to be able to, for example, set an environment variable in the script that’s called when someone loads the spark module. That way it would “just work”, and individual users wouldn’t need to remember to set it in their script, or even in their .bashrc.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 20, 2017, 2:12pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/6 "2017-10-20T14:12:17Z")

</div>

I created [a PR](https://github.com/hail-is/hail/pull/2327) to address this in our development branch. I went with `TMPDIR` as its in the POSIX standard.

Unfortunately, this is a breaking change to our python interface so I can’t back port this change to 0.1.

One might argue this is actually a bug fix, given that we ignore a standardized environment variable. I’ll poll the community over gitter.

---

<div class="post-metadata">

**Author:** ![rca](https://avatars.discourse-cdn.com/v4/letter/r/9fc29f/32.png) [@rca](https://discuss.hail.is/u/rca)\
**Post date:** [October 25, 2017, 7:23pm UTC](https://discuss.hail.is/t/hailcontext-tmp-dir/323/7 "2017-10-25T19:23:10Z")

</div>

Thanks for making this change 👍