# Importing multiple vcfs from multiple Terra buckets to Hail

**URL:** <https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259>\
**Category:** Hail Batch & General Cloud\
**Created:** [October 26, 2026, 8:44pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259 "2026-10-26T20:44:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tetyana\_Zayats](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tetyana_zayats/32/1101_2.png) [@Tetyana\_Zayats](https://discuss.hail.is/u/Tetyana_Zayats)\
**Post date:** [October 26, 2026, 8:44pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259/1 "2026-10-26T20:44:20Z")

</div>

Hi,

I need to import multiple vcf files from multiple Terra buckets into Hail. I am using hl.import\_vcf and specify the path to vcfs with a wildcard, it looks like this:  
`vcfs='gs://bucket-name/submissions/folder-name/ReblockGVCF/**/*.vcf.gz'`

I get the error that “arguments refer to no files”. It seems that Hail treats the entry as a vector.

Overall, I’ve got a manifest with a list of vcf files and paths to them (in various buckets and various folders within those buckets) and I am looking for the best way to import them. For now, I’ve tried one bucket and put the path common to the files I need, replacing the different parts of the path with wild card(s). It did not work (see the error above). Any suggestions on how to address the error/ best way to import the files are very much appreciated! Thank you!

---

<div class="post-metadata">

**Author:** ![chrisvittal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/chrisvittal/32/327_2.png) [@chrisvittal](https://discuss.hail.is/u/chrisvittal)\
**Post date:** [October 26, 2026, 9:07pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259/2 "2026-10-26T21:07:32Z")

</div>

Hi @Tetyana_Zayats, thanks for reaching out. The issue here is that unlike `gcloud storage ls` or shell expansion, [Hadoop Glob Patterns](https://hail.is/docs/0.2/hadoop_glob_patterns.html#sec-hadoop-glob) don’t expand `**` into any number of path segments separated by `/`.

You’ll need to build the file list manually. My recommendation is to use `gcloud storage ls` with those patterns, saving the list to a file.

```sh
# remove the list so we don't keep appending to it, if something went
# wrong and it needs to be regenerated
rm filelist.txt
prefixes="gs://bucket-name0/...
gs://bucket-name1/..."
for path in $prefixes; do
  # use this funny quoting so the glob gets passed along
  # to gcloud and not the shell
  gcloud storage ls "$path"'ReblockGVCF/**/*.vcf.gz' \
    >> filelist.txt
done

```

Then, in python:

```python
with open('filelist.txt') as filelist:
    vcfs = [file.strip() for file in filelist]

```

---

<div class="post-metadata">

**Author:** ![Tetyana\_Zayats](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tetyana_zayats/32/1101_2.png) [@Tetyana\_Zayats](https://discuss.hail.is/u/Tetyana_Zayats)\
**Post date:** [October 26, 2026, 9:26pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259/3 "2026-10-26T21:26:19Z")

</div>

Thank you! I will give it a go!

Because there are several buckets with files I need, can I generate one file list and then ask hail to copy those from different buckets? Or Do I need to do this for each bucket individually?

---

<div class="post-metadata">

**Author:** ![chrisvittal](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/chrisvittal/32/327_2.png) [@chrisvittal](https://discuss.hail.is/u/chrisvittal)\
**Post date:** [October 26, 2026, 9:31pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259/4 "2026-10-26T21:31:50Z")

</div>

No, you should be able to list all the buckets and get everything into one file, then assuming your user/service accounts have appropriate access, handle everything without extra copies.

---

<div class="post-metadata">

**Author:** ![Tetyana\_Zayats](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tetyana_zayats/32/1101_2.png) [@Tetyana\_Zayats](https://discuss.hail.is/u/Tetyana_Zayats)\
**Post date:** [October 26, 2026, 10:04pm UTC](https://discuss.hail.is/t/importing-multiple-vcfs-from-multiple-terra-buckets-to-hail/4259/5 "2026-10-26T22:04:04Z")

</div>

Thank you!  
I’ll give it a go, although I’m not entirely sure I understand everything just yet. Hopefully, things will become clearer as I work through the suggested steps. I’ll follow up, if I run into any trouble.  
Thank you for your help!