# Failed to install Hail on Azure following the tutorial

**URL:** <https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559>\
**Category:** Hail Query & hailctl\
**Created:** [July 29, 2020, 6:28pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559 "2020-07-29T18:28:15Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [July 29, 2020, 6:28pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/1 "2020-07-29T18:28:15Z")

</div>

I followed [this tutorial](https://github.com/TheEagleByte/azure-hail) and everything seems to be okay until I came to Step 21.

> “Save the changes and restart the affected services. These changes need a restart of Spark2 service. Ambari UI will prompt a required restart reminder, click Restart to restart all affected services.”

After I initiated the restart, one of the items in the restart dialog (“run\_customscriptaction”) seems to have failed.

I clicked into the “Run Customscriptaction Actionexecute” screen and this is the error log:

 ![d](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/b92d7042c743f84223264cab9a2ff520c2a07e69.png)

The stdout is quite large, but the bottom of it looks like this

```python
statsmodels-0. 99% |############################## | Time: 0:00:00 76.77 MB/s
statsmodels-0. 99% |############################## | Time: 0:00:00 76.78 MB/s
statsmodels-0. 99% |############################## | Time: 0:00:00 76.80 MB/s
statsmodels-0. 100% |###############################| Time: 0:00:00 76.80 MB/s
statsmodels-0. 100% |###############################| Time: 0:00:00 76.72 MB/s
anaconda-2020. 0% | | ETA: --:--:-- 0.00 B/s
anaconda-2020. 100% |###############################| Time: 0:00:00 39.12 MB/s
anaconda-2020. 100% |###############################| Time: 0:00:00 27.61 MB/s
('Start downloading script locally: ', u'https://raw.githubusercontent.com/TheEagleByte/azure-hail/master/hail-install.sh')
Fromdos line ending conversion successful
('Unexpected error:', "('Execution of custom script failed with exit code', 1)")
Removing temp location of the script

Command failed after 1 tries

```

It seems that the installation script is looking for a particular pip that was not installed. Do you think a misconfiguration causes this?

Thanks a lot!

@kumarveerapen

---

<div class="post-metadata">

**Author:** ![kumarveerapen](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/kumarveerapen/32/355_2.png) [@kumarveerapen](https://discuss.hail.is/u/kumarveerapen)\
**Post date:** [July 29, 2020, 6:31pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/2 "2020-07-29T18:31:12Z")

</div>

I wonder if @IqviaGarrettBromley could chime in on this since he has had experience in setting up Hail on Azure.

---

<div class="post-metadata">

**Author:** ![IqviaGarrettBromley](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/iqviagarrettbromley/32/375_2.png) [@IqviaGarrettBromley](https://discuss.hail.is/u/IqviaGarrettBromley)\
**Post date:** [July 30, 2020, 6:00pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/3 "2020-07-30T18:00:42Z")

</div>

Hi @xunzhustjude,

Can you share a screenshot or something of the script actions page for the cluster in the Azure dashboard? That is where they should have executed and displayed a status of successful/failed.

Also, at the moment, not sure I would consider setting up in Azure, as there is a known issue where you are unable to write any hail table or matrix table to the ADLS Gen 2 storage. You will receive errors regarding “the stream has already been closed”. The last working version on Azure is 0.2.34, and with 0.2.35 there were ~2.6k lines of changes around the `io` package where the bug was introduced.

If you have the option to, I would recommend moving to a different cloud provider, or use that version (which I wouldn’t necessarily recommend).

---

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [July 30, 2020, 6:39pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/4 "2020-07-30T18:39:23Z")

</div>

Thanks for your help, @IqviaGarrettBromley!

> Can you share a screenshot or something of the script actions page for the cluster in the Azure dashboard?

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/16f07b6b29f8214e3a58a0a49208f0900213beaf.png)

---

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [July 30, 2020, 6:39pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/5 "2020-07-30T18:39:55Z")

</div>

When clicking into the failed action:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/75acc5d257287b33155be46153b1ab54220632ee.png)

---

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [July 30, 2020, 6:40pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/6 "2020-07-30T18:40:40Z")

</div>

I attached the entire debug information here: [debug\_information.txt](https://discuss.hail.is/uploads/short-url/dYGc9qNzJQQz4SFRAaEWvpqhD3G.txt) (6.2 KB)

> If you have the option to, I would recommend moving to a different cloud provider, or use that version (which I wouldn’t necessarily recommend).

Currently Azure is what we prefer because it is a close collaborator of St. Jude. However, I will discuss with our team members regarding the possibility of switching to another cloud provider for Hail.

* * *

Sorry for making multiple posts, but it seems that as a new user I am not able to upload more than one picture per post.

---

<div class="post-metadata">

**Author:** ![IqviaGarrettBromley](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/iqviagarrettbromley/32/375_2.png) [@IqviaGarrettBromley](https://discuss.hail.is/u/IqviaGarrettBromley)\
**Post date:** [July 30, 2020, 8:44pm UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/7 "2020-07-30T20:44:19Z")

</div>

Thanks for the info. I just used that script last month, so not sure why it’s suddenly throwing an error. Can you show what the cluster configuration looks like? Versions and storage options and such?

---

<div class="post-metadata">

**Author:** ![kumarveerapen](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/kumarveerapen/32/355_2.png) [@kumarveerapen](https://discuss.hail.is/u/kumarveerapen)\
**Post date:** [August 7, 2020, 1:55am UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/8 "2020-08-07T01:55:19Z")

</div>

Has this been solved? 🙂

---

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [August 7, 2020, 2:50am UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/9 "2020-08-07T02:50:03Z")

</div>

@IqviaGarrettBromley

Thanks for helping!

The Cluster type as shown on Azure is: Spark 2.4 (HDI 4.0)

Here are the rest of the versions

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/53cc14cdf0742aa2fe26570f955bc95a43416f03.png)

Which “configuration” file do you need?

---

<div class="post-metadata">

**Author:** ![xunzhustjude](https://avatars.discourse-cdn.com/v4/letter/x/a6a055/32.png) [@xunzhustjude](https://discuss.hail.is/u/xunzhustjude)\
**Post date:** [August 7, 2020, 2:51am UTC](https://discuss.hail.is/t/failed-to-install-hail-on-azure-following-the-tutorial/1559/10 "2020-08-07T02:51:06Z")

</div>

@kumarveerapen Sorry Kumar, I was distracted by something else for the past few days. I’m coming back to this now.
