Are these prescriptions still valid for hail on terra? I query the environment to learn the project id and use
hl.init(spark_conf={
'spark.hadoop.fs.gs.requester.pays.mode': 'CUSTOM',
'spark.hadoop.fs.gs.requester.pays.buckets': 'gs://ukb-diverse-pops-public',
'spark.hadoop.fs.gs.requester.pays.project.id': 'terra-c9c997fd'
})
to initialize.
Then
>>> hl.read_matrix_table('gs://ukb-diverse-pops-public/sumstats_release/results_full.mt').describe()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<decorator-gen-1398>", line 2, in read_matrix_table
File "/opt/conda/lib/python3.7/site-packages/hail/typecheck/check.py", line 577, in wrapper
return __original_func(*args_, **kwargs_)
File "/opt/conda/lib/python3.7/site-packages/hail/methods/impex.py", line 2469, in read_matrix_table
for rg_config in Env.backend().load_references_from_dataset(path):
File "/opt/conda/lib/python3.7/site-packages/hail/backend/spark_backend.py", line 337, in load_references_from_dataset
return json.loads(self.hail_package().variant.ReferenceGenome.fromHailDataset(self.fs._jfs, path))
File "/opt/conda/lib/python3.7/site-packages/py4j/java_gateway.py", line 1322, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/opt/conda/lib/python3.7/site-packages/hail/backend/py4j_backend.py", line 31, in deco
raise fatal_error_from_java_error_triplet(deepest, full, error_id) from None
hail.utils.java.FatalError: GoogleJsonResponseException: 400 Bad Request
GET https://storage.googleapis.com/storage/v1/b/ukb-diverse-pops-public/o/sumstats_release%2Fresults_full.mt?fields=bucket,name,timeCreated,updated,generation,metageneration,size,contentType,contentEncoding,md5Hash,crc32c,metadata
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Bucket is a requester pays bucket but no user project provided.",
"reason" : "required"
} ],
"message" : "Bucket is a requester pays bucket but no user project provided."
}