I am using Hail to read vcfs from a S3 bucket. I init hail and then import the vcfs from S3a. All is ok for maybe two or three hours of analysis, but then the connection is lost and output an error. If I try to stop and init the Hail each time I import a vcf then is ok but as the analysis is distributed and needs to create a lot of files I finally get a too many open files and also the performance is lower and lower. So, I don’t know if you have worked with S3 directly and how you achieved.