# Static plotting or dataframe extraction

**URL:** https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984
**Category:** Hail Query & hailctl
**Created:** [June 12, 2019, 6:43pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984 "2019-06-12T18:43:46Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![jakewendt](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/jakewendt/32/282_2.png) [@jakewendt](https://discuss.hail.is/u/jakewendt)
#### Post date: [June 12, 2019, 6:43pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/1 "2019-06-12T18:43:46Z")

</div>

I have just started investigating the use of Hail for GWAS analysis. I created a jupyter notebook of your GWAS demo and everything seemed to go well, locally. However, when I saved this notebook to my github repository, I noticed that the plots did not render. I see that this has to do with Bokeh’s use of javascript to load the images and Github’s disabling of javascript functionality. I’ve seen some workarounds to this that suggest loading INLINE, but have not been able to successfully implement them. In addition, I see no way mentioned to programmatically save/export these plots as png when hail is used outside of a notebook. I do see a save icon, along with others, next to each plot in the notebook. I also see that bokeh has some ways to save or export plots as pngs, but they are all failing in their own way.

The interactiveness of the bokeh plots is nice and I’d like to continue using Github to share my notebooks but I’d like to have access to the raw plots as images to include in other documents and potential publications.

Is there a way to render and/or save these plots that I’m not seeing?  
Can hail have the option to use matplotlib to plot instead of bokeh?  
Can these data be extracted from the hail objects and into a dataframe or similar, then plotted with matplotlib?

I’m guessing that the latter is likely the simplest, but I still haven’t figured it out.

Thanks in advance,  
Jake

---

<div class="post-metadata">

### Author: ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 12, 2019, 7:01pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/2 "2019-06-12T19:01:48Z")

</div>

> Is there a way to render and/or save these plots that I’m not seeing?

These [bokeh functions](https://bokeh.pydata.org/en/latest/docs/reference/io.html) should work to export plots, I think. How are these failing?

I’m not totally sure how GitHub’s renderer works – the GWAS tutorial in our documentation is generated using [nbconvert](https://nbconvert.readthedocs.io/en/latest/) to export HTML, which seems to work fine.

> Can hail have the option to use matplotlib to plot instead of bokeh?

We do intend to build multiple plotting backends, but it’s not a high priority right now. It feels like we really need to build our own plotting dialect which the Hail plotting functionality rests on, then build backends for that in bokeh, plotly, matplotlib, etc.

Can these data be extracted from the hail objects and into a dataframe or similar, then plotted with matplotlib?

That’s probably the first step to building support for multiple backends. We generally do this internally:

> <https://github.com/hail-is/hail/blob/b874f6ad163ee9a8a2056707363f5d4aa08679f3/hail/python/hail/plot/plots.py#L920>

Again, not a super high priority to address right now, but pull requests are welcome!

---

<div class="post-metadata">

### Author: ![jakewendt](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/jakewendt/32/282_2.png) [@jakewendt](https://discuss.hail.is/u/jakewendt)
#### Post date: [June 12, 2019, 9:29pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/3 "2019-06-12T21:29:11Z")

</div>

Thank you for the quick response. It is my understanding the github escapes all javascript from the file prior to rendering so the bokeh javascript never runs so the images never load. Plots that I’ve previously generated with matplotlib are included inline as pngs with the notebook so no javascript is required to render them. Of course, I don’t believe that they are interactive as are the bokeh plots.

“Failing” is not really a fair assessment.

Let’s say

> p = hl.plot.manhattan(gwas.p\_value)

from the tutorial.

The following is interesting. Producing an html page complete with all javascript and images encoded in the html text. Not terribly helpful for me.

> import bokeh.plotting  
> bokeh.plotting.output\_file(“all\_of\_my\_plots.HTML”)  
> bokeh.plotting.save( p )

Both of the following styles of image access fail with respect to phantomjs.

> from bokeh.io.export import get\_screenshot\_as\_png  
> from selenium import webdriver  
> image = get\_screenshot\_as\_png(p, height=100, width=300, driver=webdriver)

yields …

> AttributeError: module ‘selenium.webdriver’ has no attribute ‘get’

The above is likely my fault as I need to better understand what to use as “webdriver”

And finally, after …

> pip install --upgrade --user pillow selenium  
> sudo port install npm6  
> sudo npm install -g phantomjs-prebuilt --ignore-scripts

> from [bokeh.io](http://bokeh.io) import export\_png  
> export\_png(p, filename=“manhattan\_plot.png”)

it fails with …

> RuntimeError: Error encountered in PhantomJS detection: ‘internal/validators.js:125\n throw new ERR\_INVALID\_ARG\_TYPE(name, 'string', value);\n ^\n\nTypeError [ERR\_INVALID\_ARG\_TYPE]: The “file” argument must be of type string. Received type object\n at validateString (internal/validators.js:125:11)\n at normalizeSpawnArguments (child\_process.js:411:3)\n at spawn (child\_process.js:545:16)\n at Object. (/opt/local/lib/node\_modules/phantomjs-prebuilt/bin/phantomjs:22:10)\n at Module.\_compile (internal/modules/cjs/loader.js:776:30)\n at Object.Module.\_extensions…js (internal/modules/cjs/loader.js:787:10)\n at Module.load (internal/modules/cjs/loader.js:653:32)\n at tryModuleLoad (internal/modules/cjs/loader.js:593:12)\n at Function.Module.\_load (internal/modules/cjs/loader.js:585:3)\n at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)\n’

Again, “fail” isn’t really accurate. This is more of a lack of understanding in the usage at this point.

Eventually, I’ll put down my hammer and figure it out.

Thanks again,  
Jake

---

<div class="post-metadata">

### Author: ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 12, 2019, 9:44pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/4 "2019-06-12T21:44:47Z")

</div>

I think part of the problem is that we fix our bokeh dependency to an ancient version – Hail 0.2.15 (to be released in the next day or two) updates to the latest.

---

<div class="post-metadata">

### Author: ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 12, 2019, 9:45pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/5 "2019-06-12T21:45:05Z")

</div>

For reference, I also now remember hitting those same issues with bokeh, though things are working now.

---

<div class="post-metadata">

### Author: ![jakewendt](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/jakewendt/32/282_2.png) [@jakewendt](https://discuss.hail.is/u/jakewendt)
#### Post date: [June 13, 2019, 10:37pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/6 "2019-06-13T22:37:56Z")

</div>

Understood. Thank you. I’ll wait until this release before I dig any deeper.

---

<div class="post-metadata">

### Author: ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 14, 2019, 8:19pm UTC](https://discuss.hail.is/t/static-plotting-or-dataframe-extraction/984/7 "2019-06-14T20:19:19Z")

</div>

(it’s out)
