Open notebooks causes 500 Internal Server Error in Dataproc

Hail Query & hailctl
1

Hi Hail team,

I am writing to get advice on Dataproc issue. I run Hail on Dataproc on GCP.
Today, I got “500 internal server error”, which is that I have never seen before until yesterday when I try to open a Jupyter notebook. If you have any idea to solve it, please share with me. Thank you.

Screen Shot 2021-12-06 at 2.22.28 PM
Screen Shot 2021-12-06 at 2.22.28 PM2400×698 72.7 KB

Here is my code to launch Dataproc and open the jupyter notebook

hailctl dataproc start ${cluster_name} --vep GRCh38 \
        --requester-pays-allow-annotation-db \
        --packages gnomad --requester-pays-allow-buckets gnomad-public-requester-pays \
        --master-machine-type=n1-highmem-8 --worker-machine-type=n1-highmem-8 \
        --num-workers=2 --num-secondary-workers=0 \
        --worker-boot-disk-size=1000

hailctl dataproc connect ${cluster_name} notebook

-Jina

2

Can you replicate this by creating a fresh cluster and trying to connect, or is it a particular cluster that has this problem?

3

Hi Tim.

Thank you for your supports. The issue has been resolved by updating Hail from v0.2.62 to 0.2.78. :grinning:

Best,
Jina

4

0.2.79 also made the hail error messages from dataproc better, no reason not to upgrade one more :+1: