# How to deploy the latest Hail 0.2 on AWS EMR?
**URL:** https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960
**Category:** Hail Query & hailctl
**Created:** [March 15, 2021, 8:46pm UTC](https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960 "2021-03-15T20:46:30Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@nickzren](https://discuss.hail.is/u/nickzren)
#### Post date: [March 15, 2021, 8:46pm UTC](https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960/1 "2021-03-15T20:46:30Z")
I have already tried two tutorials below:
> **[aws-quickstart/quickstart-hail](https://github.com/aws-quickstart/quickstart-hail)**
>
> AWS Quick Start Team. Contribute to aws-quickstart/quickstart-hail development by creating an account on GitHub.
> **[hms-dbmi/hail-on-AWS-spot-instances](https://github.com/hms-dbmi/hail-on-AWS-spot-instances)**
>
> An option to spin cost effective EMR clusters in AWS with Hail and JupyterNotebook installed
I can deploy older version hail 0.2 via quickstart-hail, however when tried to build with latest version it failed.
Then I have tried hail-on-AWS-spot-instances, but it failed to build the latest hail too.
I have submitted tickets to both repo, but did not get response, maybe they no longer maintain it.
Does anyone has an up to date tutorial to setup the latest Hail 0.2 in AWS EMR? Thanks
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [March 16, 2021, 2:22pm UTC](https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960/2 "2021-03-16T14:22:13Z")
I don’t know of any such resources right now. What’s the error you’re seeing with the hms-dbmi build?
---
### Author:  [@nickzren](https://discuss.hail.is/u/nickzren)
#### Post date: [March 16, 2021, 3:22pm UTC](https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960/3 "2021-03-16T15:22:05Z")
Here I copied errors from their auto generated `cloudcreation_log.out`:
> cd build/deploy; python3 setup.py -q sdist bdist\_wheel
> Traceback (most recent call last):
> File “/usr/lib64/python3.6/distutils/core.py”, line 148, in setup
> dist.run\_commands()
> File “/usr/lib64/python3.6/distutils/dist.py”, line 955, in run\_commands
> self.run\_command(cmd)
> File “/usr/lib64/python3.6/distutils/dist.py”, line 974, in run\_command
> cmd\_obj.run()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/bdist\_egg.py”, line 152, in run
> self.run\_command(“egg\_info”)
> File “/usr/lib64/python3.6/distutils/cmd.py”, line 313, in run\_command
> self.distribution.run\_command(command)
> File “/usr/lib64/python3.6/distutils/dist.py”, line 974, in run\_command
> cmd\_obj.run()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/egg\_info.py”, line 280, in run
> self.find\_sources()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/egg\_info.py”, line 295, in find\_sources
> mm.run()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/egg\_info.py”, line 526, in run
> self.add\_defaults()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/egg\_info.py”, line 562, in add\_defaults
> sdist.add\_defaults(self)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/py36compat.py”, line 34, in add\_defaults
> self.\_add\_defaults\_python()
> File “/usr/lib/python3.6/dist-packages/setuptools/command/sdist.py”, line 134, in \_add\_defaults\_python
> self.filelist.extend(build\_py.get\_source\_files())
> File “/usr/lib64/python3.6/distutils/command/build\_py.py”, line 301, in get\_source\_files
> return [module[-1] for module in self.find\_all\_modules()]
> File “/usr/lib64/python3.6/distutils/command/build\_py.py”, line 296, in find\_all\_modules
> m = self.find\_package\_modules(package, package\_dir)
> File “/usr/lib64/python3.6/distutils/command/build\_py.py”, line 218, in find\_package\_modules
> self.check\_package(package, package\_dir)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/build\_py.py”, line 163, in check\_package
> init\_py = orig.build\_py.check\_package(self, package, package\_dir)
> File “/usr/lib64/python3.6/distutils/command/build\_py.py”, line 191, in check\_package
> “package directory ‘%s’ does not exist” % package\_dir)
> distutils.errors.DistutilsFileError: package directory ‘find\_namespace:’ does not exist
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 158, in save\_modules
> yield saved
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 199, in setup\_context
> yield
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 254, in run\_setup
> \_execfile(setup\_script, ns)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 49, in \_execfile
> exec(code, globals, locals)
> File “/tmp/easy\_install-u9a2azbv/pytest-runner-5.3.0/setup.py”, line 21, in
> description=“Scalable library for exploring and analyzing genomic data.”,
> File “/usr/lib64/python3.6/distutils/core.py”, line 163, in setup
> raise SystemExit("error: " + str(msg))
> SystemExit: error: package directory ‘find\_namespace:’ does not exist
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 1123, in run\_setup
> run\_setup(setup\_script, args)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 257, in run\_setup
> raise
> File “/usr/lib64/python3.6/contextlib.py”, line 99, in **exit**
> self.gen.throw(type, value, traceback)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 199, in setup\_context
> yield
> File “/usr/lib64/python3.6/contextlib.py”, line 99, in **exit**
> self.gen.throw(type, value, traceback)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 170, in save\_modules
> saved\_exc.resume()
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 145, in resume
> six.reraise(type, exc, self.\_tb)
> File “/usr/lib/python3.6/dist-packages/pkg\_resources/\_vendor/six.py”, line 685, in reraise
> raise value.with\_traceback(tb)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 158, in save\_modules
> yield saved
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 199, in setup\_context
> yield
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 254, in run\_setup
> \_execfile(setup\_script, ns)
> File “/usr/lib/python3.6/dist-packages/setuptools/sandbox.py”, line 49, in \_execfile
> exec(code, globals, locals)
> File “/tmp/easy\_install-u9a2azbv/pytest-runner-5.3.0/setup.py”, line 21, in
> description=“Scalable library for exploring and analyzing genomic data.”,
> File “/usr/lib64/python3.6/distutils/core.py”, line 163, in setup
> raise SystemExit("error: " + str(msg))
> SystemExit: error: package directory ‘find\_namespace:’ does not exist
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
> File “setup.py”, line 49, in
> tests\_require=[“pytest”]
> File “/usr/lib64/python3.6/distutils/core.py”, line 108, in setup
> \_setup\_distribution = dist = klass(attrs)
> File “/usr/lib/python3.6/dist-packages/setuptools/dist.py”, line 325, in **init**
> self.fetch\_build\_eggs(attrs[‘setup\_requires’])
> File “/usr/lib/python3.6/dist-packages/setuptools/dist.py”, line 446, in fetch\_build\_eggs
> replace\_conflicting=True,
> File “/usr/lib/python3.6/dist-packages/pkg\_resources/ **init**.py”, line 855, in resolve
> dist = best[req.key] = env.best\_match(req, ws, installer)
> File “/usr/lib/python3.6/dist-packages/pkg\_resources/ **init**.py”, line 1127, in best\_match
> return self.obtain(req, installer)
> File “/usr/lib/python3.6/dist-packages/pkg\_resources/ **init**.py”, line 1139, in obtain
> return installer(requirement)
> File “/usr/lib/python3.6/dist-packages/setuptools/dist.py”, line 518, in fetch\_build\_egg
> return cmd.easy\_install(req)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 691, in easy\_install
> return self.install\_item(spec, dist.location, tmpdir, deps)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 717, in install\_item
> dists = self.install\_eggs(spec, download, tmpdir)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 898, in install\_eggs
> return self.build\_and\_install(setup\_script, setup\_base)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 1137, in build\_and\_install
> self.run\_setup(setup\_script, setup\_base, args)
> File “/usr/lib/python3.6/dist-packages/setuptools/command/easy\_install.py”, line 1125, in run\_setup
> raise DistutilsError(“Setup script exited with %s” % (v.args[0],))
> distutils.errors.DistutilsError: Setup script exited with error: package directory ‘find\_namespace:’ does not exist
> make: \*\*\* [build/deploy/dist/hail-0.2.53-py3-none-any.whl] Error 1
> ls: cannot access /opt/hail-on-AWS-spot-instances/src/hail/hail/build/deploy/dist: No such file or directory
> ERROR: Invalid requirement: ‘/opt/hail-on-AWS-spot-instances/src/hail/hail/build/deploy/dist/’
> Hint: It looks like a path. File ‘/opt/hail-on-AWS-spot-instances/src/hail/hail/build/deploy/dist/’ does not exist.
---
### Author:  [@hmkim](https://discuss.hail.is/u/hmkim)
#### Post date: [July 11, 2024, 5:17am UTC](https://discuss.hail.is/t/how-to-deploy-the-latest-hail-0-2-on-aws-emr/1960/4 "2024-07-11T05:17:34Z")
See this repo. With EMR 6.x, it’s easier than ever to provision a Jupyter environment that can run hail through Amazon EMR and SageMaker notebook. Automate AMI builds, including installing Hail and VEP.
> **[GitHub - hmkim/quickstart-hail: Hail on AWS EMR Sample](https://github.com/hmkim/quickstart-hail)**
>
> Hail on AWS EMR Sample. Contribute to hmkim/quickstart-hail development by creating an account on GitHub.