I have no problem using export_vcf on a normal MatrixTable after filtering, but when i use vep to annotate the file and apply the same function, it gives me an IO:Error, file or directory does not exist.
I have checked and the directory to which I try to write does exist.
What is wrong?
KR Mathias.
What’s the full stack trace? This is probably an error related to VEP, not write.
What VEP config are you using?
I use a modified version of the one here:
https://hail.is/docs/0.2/methods/genetics.html?highlight=vep#hail.methods.vep
I realised that I forgot to edit /vep to my own version, just vep. Unfortunately my changes in the configuration file has not solved the problem, see my current error message. The command works fine outside of hail.
KR
You can use VEP like that on the machines running the Hail worker processes, not just the driver process, right?
I am not sure what you mean, but I am able to source the Hail environment and then successfully run the command (in command line) seen in the screenshot without errors (vep --format vcf …). I should mention that i add the “-i vcf” variable.
Any suggestions?
I think it should be possible to print VEP’s error output in this error message. I’ll look into that today.
Sorry, quite busy today. I should be able to look tomorrow afternoon.
and?
Thanks for the reminder - working through a backlog after a group retreat Mon/Tues.
I’ve opened this pull request but I’m not totally sure it will work. I found this hard to test when I poked around at it last week.
any news?
that pull request was merged 16 days ago. Can you try again?
After updating the same error occurs. Another thing: it seems like the update for some reason has introduced a bug so that visually the job doesn’t update fluently in the GUI (jupyter notebook). The process looked like it was stuck running a simple function for hours. I actually thought there was an error in the function, but returning to jupyter today, I see that the all of my workflow completed (until the error).
KR
You mean the jupyter-spark progress bar? The Spark progress bar and everything related is super finicky.
To clarify, you’re still getting the “VEP returned error code 2” error, right?
I get that error, yes. I mean that when I run Hail in jupyter-notebook, the progress of each block of code that I run, doesn’t seem to update accordingly to the actually completed blocks.
How are you measuring this? With the [*] icons in the cells, or the jupyter spark progress bar?
The [*] icon.
Any progress?