Hi, I have a custom python generator that’s pulling data out of a hail blockmatrix to use for deep learning, as previously discussed here . This has been working pretty well by itself, but when I try to parallelise the data generator (by running multiple workers), I get the following error: -------…

Accessing Blockmatrix data in parallel

amanas September 5, 2022, 7:34am 13

because you are working with Pytorch, maybe this Feature Request could be of interest.

Regards,
Andrés.

Topic		Replies	Views
SNP dosages to numpy/pandas? Hail Query & hailctl	23	1724	September 5, 2022
HailException: Cannot create BlockMatrix: Hail Query & hailctl	2	406	February 19, 2020
Small MatrixTable hangs on write into Google bucket Hail Query & hailctl	13	969	September 5, 2019
py4j.protocol.Py4JError: is.hail.variant does not exist in the JVM Hail Query & hailctl	1	1130	March 23, 2022
Ukbb_common documentation seems stale Feature Requests	12	472	January 21, 2023