# Not able to write to vds

**URL:** <https://discuss.hail.is/t/not-able-to-write-to-vds/269>\
**Category:** Help \[0.1\]\
**Created:** [August 25, 2017, 6:46pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269 "2017-08-25T18:46:34Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [August 25, 2017, 6:46pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/1 "2017-08-25T18:46:34Z")

</div>

Hi,

I am able to import vcf by  
vds = hc.import\_vcf(‘1kg.vcf.bgz’)  
and I am able to get the summarize report by  
vds.summarize().report()

however, i got the following error when I tried to write vds. Could you help?

Thanks,  
Wendy

> > > vds.write(‘test.vds’)  
> > > Traceback (most recent call last):  
> > > File “”, line 1, in   
> > > File “”, line 2, in write  
> > > File “build/distributions/hail-python.zip/hail/java.py”, line 121, in handle\_py4j  
> > > hail.java.FatalError: ClassNotFoundException: is.hail.utils.SerializableHadoopConfiguration

Java stack trace:  
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 3.0 failed 4 times, most recent failure: Lost task 0.3 in stage 3.0 (TID 9, , executor 1): java.io.IOException: com.esotericsoftware.kryo.KryoException: Error during Java deserialization.  
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1276)  
at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value$lzycompute(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)  
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:287)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:279)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)  
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)  
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)  
at org.apache.spark.scheduler.Task.run(Task.scala:99)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at java.lang.Thread.run(Thread.java:748)  
Caused by: com.esotericsoftware.kryo.KryoException: Error during Java deserialization.  
at com.esotericsoftware.kryo.serializers.JavaSerializer.read(JavaSerializer.java:65)  
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)  
at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:244)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$9.apply(TorrentBroadcast.scala:290)  
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1303)  
at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:291)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)  
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)  
… 18 more  
Caused by: java.lang.ClassNotFoundException: is.hail.utils.SerializableHadoopConfiguration  
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)  
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)  
at java.lang.Class.forName0(Native Method)  
at java.lang.Class.forName(Class.java:348)  
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:677)  
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)  
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)  
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)  
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)  
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)  
at com.esotericsoftware.kryo.serializers.JavaSerializer.read(JavaSerializer.java:63)  
… 25 more

Driver stacktrace:  
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1435)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1423)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1422)  
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)  
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)  
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1422)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)  
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:802)  
at scala.Option.foreach(Option.scala:257)  
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:802)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1650)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1605)  
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1594)  
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)  
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:628)  
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1918)  
at org.apache.spark.SparkContext.runJob(SparkContext.scala:1981)  
at org.apache.spark.rdd.RDD$$anonfun$fold$1.apply(RDD.scala:1088)  
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)  
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)  
at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)  
at org.apache.spark.rdd.RDD.fold(RDD.scala:1082)  
at is.hail.annotations.RichRDDUnsafeRow$.writeRows$extension(ReadRowsRDD.scala:307)  
at is.hail.variant.VariantSampleMatrix.write(VariantSampleMatrix.scala:2071)  
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
at java.lang.reflect.Method.invoke(Method.java:498)  
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)  
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)  
at py4j.Gateway.invoke(Gateway.java:280)  
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)  
at py4j.commands.CallCommand.execute(CallCommand.java:79)  
at py4j.GatewayConnection.run(GatewayConnection.java:214)  
at java.lang.Thread.run(Thread.java:748)java.io.IOException: com.esotericsoftware.kryo.KryoException: Error during Java deserialization.  
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1276)  
at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value$lzycompute(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)  
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:287)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:279)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)  
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)  
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)  
at org.apache.spark.scheduler.Task.run(Task.scala:99)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at java.lang.Thread.run(Thread.java:748)com.esotericsoftware.kryo.KryoException: Error during Java deserialization.  
at com.esotericsoftware.kryo.serializers.JavaSerializer.read(JavaSerializer.java:65)  
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)  
at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:244)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$9.apply(TorrentBroadcast.scala:290)  
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1303)  
at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:291)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)  
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)  
at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value$lzycompute(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)  
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:287)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:279)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)  
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)  
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)  
at org.apache.spark.scheduler.Task.run(Task.scala:99)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at java.lang.Thread.run(Thread.java:748)java.lang.ClassNotFoundException: is.hail.utils.SerializableHadoopConfiguration  
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)  
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)  
at java.lang.Class.forName0(Native Method)  
at java.lang.Class.forName(Class.java:348)  
at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:677)  
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)  
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)  
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)  
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)  
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)  
at com.esotericsoftware.kryo.serializers.JavaSerializer.read(JavaSerializer.java:63)  
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)  
at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:244)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$9.apply(TorrentBroadcast.scala:290)  
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1303)  
at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:291)  
at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225)  
at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1269)  
at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value$lzycompute(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.\_value(TorrentBroadcast.scala:66)  
at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)  
at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:287)  
at is.hail.annotations.RichRDDUnsafeRow$$anonfun$2.apply(ReadRowsRDD.scala:279)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$26.apply(RDD.scala:843)  
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)  
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323)  
at org.apache.spark.rdd.RDD.iterator(RDD.scala:287)  
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)  
at org.apache.spark.scheduler.Task.run(Task.scala:99)  
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
at java.lang.Thread.run(Thread.java:748)

Hail version: devel-42376b8  
Error summary: ClassNotFoundException: is.hail.utils.SerializableHadoopConfiguration

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [August 25, 2017, 7:00pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/2 "2017-08-25T19:00:46Z")

</div>

Hi Wendy,  
First, I’d really recommend using the latest 0.1 version, the `devel` version is very unstable and actually much slower than 0.1 right now. If you’re building from source, check out the `0.1` branch. You can also download the latest distribution from [this page](https://hail.is/docs/stable/getting_started.html).

I believe your error is caused by the Hail jar not being visible on the worker nodes of your Spark cluster. How are you running Spark?

---

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [August 25, 2017, 9:25pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/3 "2017-08-25T21:25:38Z")

</div>

> [@jobaba](#):
>
> vds.write(‘test.vds’)

Thank you! I am getting a pyspark2 session and run spark from there.  
I also tried spark2-submit but wasn’t sure how to pass in conf ☹

pyspark2 --jars build/libs/hail-all-spark.jar   
–py-files build/distributions/hail-python.zip   
–conf spark.sql.files.openCostInBytes=1099511627776   
–conf spark.sql.files.maxPartitionBytes=1099511627776   
–conf spark.kryo.registrator=is.hail.kryo.HailKryoRegistrator   
–conf spark.hadoop.parquet.block.size=1099511627776   
–conf spark.executor.extraClassPath=build/libs/hail-all-spark.jar   
–conf spark.driver.extraClassPath=build/libs/hail-all-spark.jar

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [August 25, 2017, 9:42pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/4 "2017-08-25T21:42:52Z")

</div>

We’ve been struggling with this internally as well. try adding these conf options:

```auto
--conf "spark.driver.extraClassPath=./hail-all-spark.jar"
--conf "spark.executor.extraClassPath=./hail-all-spark.jar"

```

---

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [August 25, 2017, 9:51pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/5 "2017-08-25T21:51:10Z")

</div>

> [@jobaba](#):
>
> vds.write(‘test.vds’)

Wow, that’s magic! it worked! Thank you soooooooooo much!!!

---

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [August 30, 2017, 9:01pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/6 "2017-08-30T21:01:19Z")

</div>

This is a different error but I thought I would keep it here since it might still have to do with Spark working nodes not being able to see the hail JAR file.

When I ran  
vds.query\_variants(‘variants.take(5)’)

I get  
Error summary: ClassNotFoundException: is.hail.asm4s.AsmFunction2

are there extra settings I can do?

Thanks,  
Wendy

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [August 31, 2017, 4:11pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/7 "2017-08-31T16:11:16Z")

</div>

Hi Wendy,  
I think you’re spot on about the jar issue. Looks like the exact same one as this post: [ClassNotFoundException: is.hail.asm4s.AsmFunction2](http://discuss.hail.is/t/classnotfoundexception-is-hail-asm4s-asmfunction2/231/6)

It seems as though I was wrong about the jar path configuration for the driver, but right about the configuration for the executor. Try this:

```python
--conf "spark.driver.extraClassPath=build/libs/hail-all-spark.jar"
--conf "spark.executor.extraClassPath=./hail-all-spark.jar"

```

---

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [September 1, 2017, 9:08pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/8 "2017-09-01T21:08:21Z")

</div>

Thanks again! The config works well this time!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [September 1, 2017, 9:09pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/9 "2017-09-01T21:09:35Z")

</div>

OK, great! This is all a big pain.

---

<div class="post-metadata">

**Author:** ![jobaba](https://avatars.discourse-cdn.com/v4/letter/j/dbc845/32.png) [@jobaba](https://discuss.hail.is/u/jobaba)\
**Post date:** [September 1, 2017, 9:10pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269/10 "2017-09-01T21:10:56Z")

</div>

No problem. Thank you for developing such a great tool!