Capturing Hail errors on DNAnexus

Hi all—

I have a quality-of-life question. When running Hail on DNAnexus, my notebook often gets disconnected while code is running (e.g. I need to close my laptop). I understand that I can use a JupyterLab magic command, e.g.

%%capture out
# a bunch of hail commands in a cell

and subsequently use out.show() to view the output. This works when the cell doesn’t produce an error. When the cell produces an error, though, out is essentially empty. It would be very useful to capture the error so that I have an idea of how to troubleshoot. Any suggestions?

I’ll post this question to the DNAnexus forums as well.

Best,
Jeremy

I think someone from DNAnexus commented that they themself never work in DNAnexus notebooks since they are unreliable. Better to work in a normal notebook and just upload it after you are done. Don’t ask me why this is the case.

Interesting, thank you! I’m not sure exactly how that would work, unless they were drafting out code locally and uploading every time they wanted to actually run something?

Ah, I misread. I thought you were talking “DNAnexus Notebooks” versus “Just Notebooks” on the cluster itself. The former which disconnects often.

FWIW, this is what we recommend. Jupyter Notebooks are just too unreliable for exactly the reasons you’re describing. I don’t know why the capture magic doesn’t pick up the exceptions. Hail’s exceptions are just normal Python exceptions :/.

Does DNANexus have a “submit script” method? This is how we generally work at Broad. We submit scripts which write to well-known locations. When the script finishes, we can kick up a notebook locally to inspect the (small) analysis results.

Thank you Dan! I’ll check in with DNAnexus about that, maybe something would be possible with their dx CLI.

1 Like