# Hail doesn't respect \`tmp\_dir\`

**URL:** <https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319>\
**Category:** Hail Query & hailctl\
**Created:** [October 28, 2021, 9:24pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319 "2021-10-28T21:24:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![hannes\_brt](https://avatars.discourse-cdn.com/v4/letter/h/3be4f8/32.png) [@hannes\_brt](https://discuss.hail.is/u/hannes_brt)\
**Post date:** [October 28, 2021, 9:24pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/1 "2021-10-28T21:24:16Z")

</div>

I have a job that keeps running out of temp disk space, so I specified the `tmp_dir` and `local_tmpdir` arguments to `hl.init()` in order to use a disk with more space. I even print the value of `hl.tmp_dir()` to check that the value has been set.

However, no matter what I set `tmp_dir` to, my job keeps failing with messages like `Error summary: FileNotFoundException: /tmp/blockmgr-209b518a-3d50-424a-a9b8-7e1b6509da6c/17/shuffle_0_32640_0.data.d76ae3c3-b5ce-4a45-ad31-671207b8329a (No space left on device)` and indeed, when I check `/tmp` while the job is running, I can see that Hail is writing lots of data there.

Is there a way to redirect _all_ temporary files to a different location?

Many thanks!

---

<div class="post-metadata">

**Author:** ![hannes\_brt](https://avatars.discourse-cdn.com/v4/letter/h/3be4f8/32.png) [@hannes\_brt](https://discuss.hail.is/u/hannes_brt)\
**Post date:** [October 29, 2021, 3:47pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/2 "2021-10-29T15:47:08Z")

</div>

I believe, I can answer this question myself:

To redirect _all_ temporary files to a particular location, you need to set `tmp_dir`, `local_tmpdir`, and also `spark.local.dir`:

```auto
temp_dir = "/my/temp/dir"

hl.init(
  tmp_dir=temp_dir,
  local_tmpdir=temp_dir,
  spark_conf={"spark.local.dir": temp_dir}
)

```

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 29, 2021, 4:25pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/3 "2021-10-29T16:25:10Z")

</div>

Hi @hannes_brt ,

Thanks for the detailed investigation. We’re discussing automatically setting the spark local dir to local\_tmpdir. That’s a pretty reasonable thing to do.

---

<div class="post-metadata">

**Author:** ![Zhiwen-Owen-Jiang](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/zhiwen-owen-jiang/32/1225_2.png) [@Zhiwen-Owen-Jiang](https://discuss.hail.is/u/Zhiwen-Owen-Jiang)\
**Post date:** [January 13, 2025, 4:09pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/4 "2025-01-13T16:09:32Z")

</div>

Hi Dan, has this problem been solved in updated versions?

---

<div class="post-metadata">

**Author:** ![ehigham](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/ehigham/32/943_2.png) [@ehigham](https://discuss.hail.is/u/ehigham)\
**Post date:** [January 21, 2025, 7:38pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/5 "2025-01-21T19:38:47Z")

</div>

Hi @Zhiwen-Owen-Jiang,  
Unfortunately not - you still need to set `spark.local.dir` yourself.  
I’ll make an effort to address this for either the upcoming release or the next.

---

<div class="post-metadata">

**Author:** ![Zhiwen-Owen-Jiang](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/zhiwen-owen-jiang/32/1225_2.png) [@Zhiwen-Owen-Jiang](https://discuss.hail.is/u/Zhiwen-Owen-Jiang)\
**Post date:** [January 21, 2025, 7:50pm UTC](https://discuss.hail.is/t/hail-doesnt-respect-tmp-dir/2319/6 "2025-01-21T19:50:42Z")

</div>

Hi ehigham, thanks for clarification. I have manually specified this in my config.