I am trying to access files on AWS S3 specified in the main VEP config json file by specifying full paths like s3://$bucket_name/$path. I verified that the files do exist on s3 at the specified paths but I am getting the error:
hail.utils.java.FatalError: IOException: error=2, No such file or directory
Regarding the file that does exist on s3 at the path that I specify in the config file:
I know that when I run locally when all the code accesses hadoop storage, in config I need to write paths to the local file system, but what is local file system in AWS then? Is it possible to run VEP from S3 storage?