# Export ElasticSearch error

**URL:** <https://discuss.hail.is/t/export-elasticsearch-error/1755>\
**Category:** Hail Query & hailctl\
**Created:** [October 28, 2020, 1:58pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755 "2020-10-28T13:58:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Santiago](https://avatars.discourse-cdn.com/v4/letter/s/3bc359/32.png) [@Santiago](https://discuss.hail.is/u/Santiago)\
**Post date:** [October 28, 2020, 1:58pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755/1 "2020-10-28T13:58:08Z")

</div>

Hi,

I am trying to export to elasticsearch but I get an error. I don’t if is for the version of ES or some configuration.

Traceback (most recent call last):  
File “”, line 1, in   
File “”, line 2, in export\_elasticsearch  
File “/usr/local/lib/python3.7/site-packages/hail/typecheck/check.py”, line 614, in wrapper  
return _original\_func(\*args, \*\*kwargs_)  
File “/usr/local/lib/python3.7/site-packages/hail/methods/impex.py”, line 2456, in export\_elasticsearch  
Env.hail().io.ElasticsearchConnector.export(jdf, host, port, index, index\_type, block\_size, config, verbose)  
File “/usr/local/lib/python3.7/site-packages/py4j/java\_gateway.py”, line 1257, in **call**  
answer, self.gateway\_client, self.target\_id, self.name)  
File “/usr/local/lib/python3.7/site-packages/hail/backend/spark\_backend.py”, line 42, in deco  
‘Error summary: %s’ % (deepest, full, hail. **version** , deepest)) from None  
hail.utils.java.FatalError: EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 7.9.3

Java stack trace:  
is.hail.relocated.org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting ‘es.nodes.wan.only’  
at is.hail.relocated.org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:327)  
at is.hail.relocated.org.elasticsearch.spark.sql.EsSparkSQL$.saveToEs(EsSparkSQL.scala:97)  
at is.hail.relocated.org.elasticsearch.spark.sql.EsSparkSQL$.saveToEs(EsSparkSQL.scala:83)  
at is.hail.relocated.org.elasticsearch.spark.sql.package$SparkDataFrameFunctions.saveToEs(package.scala:49)  
at is.hail.io.ElasticsearchConnector$.export(ElasticsearchConnector.scala:44)  
at is.hail.io.ElasticsearchConnector$.export(ElasticsearchConnector.scala:20)  
at is.hail.io.ElasticsearchConnector.export(ElasticsearchConnector.scala)  
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
at py4j.Gateway.invoke(Gateway.java:282)  
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
at py4j.commands.CallCommand.execute(CallCommand.java:79)  
at py4j.GatewayConnection.run(GatewayConnection.java:238)  
at java.lang.Thread.run(Thread.java:748)

is.hail.relocated.org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 7.9.3  
at is.hail.relocated.org.elasticsearch.hadoop.util.EsMajorVersion.parse(EsMajorVersion.java:82)  
at is.hail.relocated.org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:578)  
at is.hail.relocated.org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:320)  
at is.hail.relocated.org.elasticsearch.spark.sql.EsSparkSQL$.saveToEs(EsSparkSQL.scala:97)  
at is.hail.relocated.org.elasticsearch.spark.sql.EsSparkSQL$.saveToEs(EsSparkSQL.scala:83)  
at is.hail.relocated.org.elasticsearch.spark.sql.package$SparkDataFrameFunctions.saveToEs(package.scala:49)  
at is.hail.io.ElasticsearchConnector$.export(ElasticsearchConnector.scala:44)  
at is.hail.io.ElasticsearchConnector$.export(ElasticsearchConnector.scala:20)  
at is.hail.io.ElasticsearchConnector.export(ElasticsearchConnector.scala)  
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
at py4j.Gateway.invoke(Gateway.java:282)  
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
at py4j.commands.CallCommand.execute(CallCommand.java:79)  
at py4j.GatewayConnection.run(GatewayConnection.java:238)  
at java.lang.Thread.run(Thread.java:748)

Hail version: 0.2.57-582b2e31b8bd  
Error summary: EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 7.9.3

---

<div class="post-metadata">

**Author:** ![Santiago](https://avatars.discourse-cdn.com/v4/letter/s/3bc359/32.png) [@Santiago](https://discuss.hail.is/u/Santiago)\
**Post date:** [October 28, 2020, 3:35pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755/2 "2020-10-28T15:35:36Z")

</div>

For example now I tried directly with spark and it works… with the jar elasticsearch-hadoop-7.9.3.jar. So my question is, which jar are you using to connect to ES?

---

<div class="post-metadata">

**Author:** ![Santiago](https://avatars.discourse-cdn.com/v4/letter/s/3bc359/32.png) [@Santiago](https://discuss.hail.is/u/Santiago)\
**Post date:** [November 3, 2020, 1:31pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755/3 "2020-11-03T13:31:53Z")

</div>

Some help with this issue?. Could please tell me the elasticsearch-hadoop version that uses Hail 0.2?. Now I have to convert to spark with to\_spark but this method is really very very slow.

---

<div class="post-metadata">

**Author:** ![nawatts](https://avatars.discourse-cdn.com/v4/letter/n/839c29/32.png) [@nawatts](https://discuss.hail.is/u/nawatts)\
**Post date:** [November 4, 2020, 6:34pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755/4 "2020-11-04T18:34:54Z")

</div>

Hail uses elasticsearch-spark-20\_2.11:6.2.4 ([https://github.com/hail-is/hail/search?q=elasticsearch+spark](https://github.com/hail-is/hail/search?q=elasticsearch+spark)).

That version of ES-hadoop only supports up to Elasticsearch 6.8 ([https://www.elastic.co/support/matrix#matrix\_compatibility](https://www.elastic.co/support/matrix#matrix_compatibility)).

More info on support for Spark/Scala versions at [https://www.elastic.co/guide/en/elasticsearch/hadoop/master/install.html](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/install.html).

---

<div class="post-metadata">

**Author:** ![Santiago](https://avatars.discourse-cdn.com/v4/letter/s/3bc359/32.png) [@Santiago](https://discuss.hail.is/u/Santiago)\
**Post date:** [November 4, 2020, 6:47pm UTC](https://discuss.hail.is/t/export-elasticsearch-error/1755/5 "2020-11-04T18:47:15Z")

</div>

Ok, thank you
