I extract the column table including sample information from the Matrix table file as below.
mt_qced = hl.read_matrix_table('gs://bucket/QCed.MT') mt_qced= hl.sample_qc(mt_qced) sample_info_file='gs://bucket/sampleinfo.txt' mt_qced.cols().flatten().export(sample_info_file)
but it failed with the same error message in several runs.
Failed to connect to cluster-sw-m90f.c.project.internal:7337 at org.apache.spark.storage.ShuffleBlockFetcherIterator …
Caused by: java.io.IOException: Failed to connect to cluster-sw-m90f.c.project.internal:7337
I ran it in Dataproc in GCP with auto-scale. Please let me know any possible solutions. Thank you.