# Unable to import hail due to Numpy Scipy dependency mismatch

**URL:** <https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285>\
**Category:** Hail Query & hailctl\
**Created:** [October 6, 2021, 7:01am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285 "2021-10-06T07:01:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhishek](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/abhishek/32/570_2.png) [@Abhishek](https://discuss.hail.is/u/Abhishek)\
**Post date:** [October 6, 2021, 7:01am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/1 "2021-10-06T07:01:52Z")

</div>

Hi,  
I am currently trying to import hail(`0.2.72`). However, I get the following error.

```auto

Traceback (most recent call last): 
  File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 70, in import_module
    return importlib.import_module(modname)
  File "/usr/local/lib/python3.7/importlib/ __init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jupyter/notebooks/piranha/ __init__.py", line 11, in <module>
    module = import_module(f"{ __name__ }.{module_name}")
  File "/usr/local/lib/python3.7/importlib/ __init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jupyter/notebooks/piranha/utils.py", line 19, in <module>
    import hail as hl
  File "/usr/local/lib/python3.7/site-packages/hail/ __init__.py", line 48, in <module>
    from .methods import * # noqa: F401,F403,E402
  File "/usr/local/lib/python3.7/site-packages/hail/methods/ __init__.py", line 8, in <module>
    from .statgen import (skat, impute_sex, genetic_relatedness_matrix, realized_relationship_matrix,
  File "/usr/local/lib/python3.7/site-packages/hail/methods/statgen.py", line 16, in <module>
    from hail.linalg import BlockMatrix
  File "/usr/local/lib/python3.7/site-packages/hail/linalg/ __init__.py", line 1, in <module>
    from .blockmatrix import BlockMatrix, _jarray_from_ndarray, _breeze_from_ndarray, _svd, _eigh
  File "/usr/local/lib/python3.7/site-packages/hail/linalg/blockmatrix.py", line 7, in <module>
    import scipy.linalg as spla
  File "/usr/local/lib/python3.7/site-packages/scipy/ __init__.py", line 77, in <module>
    _fun = _deprecated(_msg.format(_key))(_fun)
  File "/usr/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 16, in wrap
    @functools.wraps(fun)
  File "/home/jupyter/notebooks/ref_docs/source/conf.py", line 27, in no_op_wraps
    if func. __module__ is None or module_name not in func. __module__ :
AttributeError: 'numpy.ufunc' object has no attribute ' __module__'

```

The versions are as follows:

```auto
pandas==1.1.4
scipy==1.6.3
numpy==1.21.1

```

PS: I installed hail(`0.2.72`) on a fresh system using `pip`.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 6, 2021, 2:52pm UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/2 "2021-10-06T14:52:22Z")

</div>

Hi @Abhishek !

I’m sorry to hear you’re having trouble with Hail. This appears to be a conflict between `skimage` and `numpy`. You should either update skimage to 0.14.2 or later:

```auto
pip install -U 'skimage>=0.14.2'

```

or downgrade numpy to 1.15:

```auto
pip install -U 'numpy<=1.15'

```

---

<div class="post-metadata">

**Author:** ![Abhishek](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/abhishek/32/570_2.png) [@Abhishek](https://discuss.hail.is/u/Abhishek)\
**Post date:** [October 8, 2021, 7:06am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/3 "2021-10-08T07:06:25Z")

</div>

Hi @danking,

These are the solutions I tried,

1. Upgrading **skimage**  
`pip install -U 'skimage>=0.14.2'`  
It didn’t work

2. Downgrading **numpy**  
`pip install -U 'numpy<=1.15'`  
It didn’t work and each time process ended up getting killed due to pip trying to figure out dependencies.

 ![Screenshot 2021-10-08 at 12.32.10 PM](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/24ec61c99a968b202e85c3ec45586dd233331848.png)

PS: To give you overall context, I have written wrapper over hail for our custom pipelines. I am trying to generate sphinx documentation. However, whenever I try to generate sphinx documentation(which requires importing hail), I get the following error.

```auto
AttributeError: 'numpy.ufunc' object has no attribute ' __module__'

```

---

<div class="post-metadata">

**Author:** ![Abhishek](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/abhishek/32/570_2.png) [@Abhishek](https://discuss.hail.is/u/Abhishek)\
**Post date:** [October 8, 2021, 7:30am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/4 "2021-10-08T07:30:06Z")

</div>

I tried installing hail `0.2.72` in a virtual environment from scratch with python `3.7` and it worked fine.  
But whenever I try to do the same in our CI/CD gitlab runner, it seems to break. I will check the gitlab runner docker base image installed packages.

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 8, 2021, 1:26pm UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/5 "2021-10-08T13:26:28Z")

</div>

Hmm. After running

```auto
pip install -U 'skimage>=0.14.2'

```

did you get this error again:

```auto
AttributeError: 'numpy.ufunc' object has no attribute ' __module__'

```

?

Hmm, OK. After a closer look, it seems that you might have changed `functools.wraps` to be a function on line 27 of `ref_docs/source/conf.py`:

```auto
  File "/home/jupyter/notebooks/ref_docs/source/conf.py", line 27, in no_op_wraps
    if func. __module__ is None or module_name not in func. __module__ :

```

It sounds like the issue might be that `numpy.ufunc` has no attribute ` __module__ `. Can you try changing your `no_op_wraps` code to be:

```auto
if not hasattr(func, ' __module__') or func. __module__ is None or module_name not in func. __module__ :

```

---

<div class="post-metadata">

**Author:** ![Abhishek](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/abhishek/32/570_2.png) [@Abhishek](https://discuss.hail.is/u/Abhishek)\
**Post date:** [October 14, 2021, 7:51am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/6 "2021-10-14T07:51:46Z")

</div>

I tried updating `no_wraps_code` as you suggested

```auto
import functools
def no_op_wraps(func):

    if hasattr(func, ' __module__'):
        if func. __module__ is None or module_name not in func. __module__ :
            return functools.orig_wraps(func)
        
    def wrapper(decorator):
        return func
    return wrapper

functools.orig_wraps = functools.wraps 
functools.wraps = no_op_wraps

```

However, I got a new error.(You can see the full stack trace at the bottom)

```auto
call. __doc__ = msg
AttributeError: attribute ' __doc__' of 'numpy.ufunc' objects is not writable

```

PS: The no op wraps code replaces functools.wraps in order to undo wrapping. It is used to preserve the decorated function’s signature in the documentation generated by Sphinx.

> **Stack trace**
>
> ```auto
> Traceback (most recent call last): 
> File "/usr/local/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 70, in import_module
> return importlib.import_module(modname)
> File "/usr/local/lib/python3.7/importlib/ __init__.py", line 127, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
> File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
> File "<frozen importlib._bootstrap>", line 983, in _find_and_load
> File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
> File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
> File "<frozen importlib._bootstrap_external>", line 728, in exec_module
> File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
> File "/home/jupyter/notebooks/piranha/ __init__.py", line 11, in <module>
> module = import_module(f"{ __name__ }.{module_name}")
> File "/usr/local/lib/python3.7/importlib/ __init__.py", line 127, in import_module
> return _bootstrap._gcd_import(name[level:], package, level)
> File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
> File "<frozen importlib._bootstrap>", line 983, in _find_and_load
> File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
> File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
> File "<frozen importlib._bootstrap_external>", line 728, in exec_module
> File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
> File "/home/jupyter/notebooks/piranha/utils.py", line 19, in <module>
> import hail as hl
> File "/usr/local/lib/python3.7/site-packages/hail/ __init__.py", line 48, in <module>
> from .methods import * # noqa: F401,F403,E402
> File "/usr/local/lib/python3.7/site-packages/hail/methods/ __init__.py", line 8, in <module>
> from .statgen import (skat, impute_sex, genetic_relatedness_matrix, realized_relationship_matrix,
> File "/usr/local/lib/python3.7/site-packages/hail/methods/statgen.py", line 16, in <module>
> from hail.linalg import BlockMatrix
> File "/usr/local/lib/python3.7/site-packages/hail/linalg/ __init__.py", line 1, in <module>
> from .blockmatrix import BlockMatrix, _jarray_from_ndarray, _breeze_from_ndarray, _svd, _eigh
> File "/usr/local/lib/python3.7/site-packages/hail/linalg/blockmatrix.py", line 7, in <module>
> import scipy.linalg as spla
> File "/usr/local/lib/python3.7/site-packages/scipy/ __init__.py", line 77, in <module>
> _fun = _deprecated(_msg.format(_key))(_fun)
> File "/usr/local/lib/python3.7/site-packages/scipy/_lib/deprecation.py", line 21, in wrap
> call. __doc__ = msg
> AttributeError: attribute ' __doc__' of 'numpy.ufunc' objects is not writable
> 
> WARNING: autodoc: failed to import module 'setup'; the module executes module level statement and it might call sys.exit(). 
> 
> ```

---

<div class="post-metadata">

**Author:** ![Abhishek](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/abhishek/32/570_2.png) [@Abhishek](https://discuss.hail.is/u/Abhishek)\
**Post date:** [October 14, 2021, 7:53am UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/7 "2021-10-14T07:53:09Z")

</div>

It seems like the issue is this: [Link](https://github.com/scipy/scipy/pull/10290)

Numpy functions were deprecated in Scipy after `1.14`

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [October 15, 2021, 4:31pm UTC](https://discuss.hail.is/t/unable-to-import-hail-due-to-numpy-scipy-dependency-mismatch/2285/8 "2021-10-15T16:31:58Z")

</div>

I agree. It appears there’s some issue between scipy and numpy. I don’t think hail is involved except that hail imports scipy. If you remove hail entirely from your utils.py file, add `import scipy.linalg` and try to generate docs, does that also fail? I’m not sure what’s wrong, but I think you’ll need to find compatible versions of scipy, numpy, and skimage.