# Exporting data from MatrixTable into TSV **URL:** \ **Category:** Hail Query & hailctl\ **Created:** [December 13, 2021, 9:56pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406 "2021-12-13T21:56:45Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\ **Post date:** [December 13, 2021, 9:56pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/1 "2021-12-13T21:56:45Z")
Hi all! I know that I am able to export a Table into a TSV file, but can I choose a few entries/rows/cols from a MatrixTable and export it into a TSV? Thank you so much! ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [December 14, 2021, 3:35pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/2 "2021-12-14T15:35:29Z")
Exporting columns to a TSV is not possible because TSVs have no obvious way to represent column metadata. You could export a single entry field to a TSV: ```auto small_mt.GT.export('output/gt.tsv') ``` If you want to include more row fields you can add them as key fields: ```auto small_mt = small_mt.key_rows_by(*mt.row_key, 'rowfield1', 'rowfield2', ...) small_mt.GT.export('output/gt-with-some-row-fields.tsv') ``` You can export all of the entry fields, but they’ll be exported as a JSON dictionary: ```auto small_mt.entry.export('output/all-entry-fields-as-json-dict.tsv') ``` More examples [here](https://hail.is/docs/0.2/hail.expr.Expression.html#hail.expr.Expression.export). * * * Aside: If you’re just trying to get the data out of Hail and into Python, you might try `localize_entries` followed by `collect`: ```auto In [9]: mt = hl.utils.range_matrix_table(2, 2) ...: mt = mt.annotate_entries(x = (mt.row_idx + 1) * (mt.col_idx + 1)) ...: t = mt.localize_entries('entries') ...: t.collect() Out[9]: [Struct(row_idx=0, entries=[Struct(x=1), Struct(x=2)]), Struct(row_idx=1, entries=[Struct(x=2), Struct(x=4)])] ``` If you’re trying to convert Hail MTs to pandas, you probably want something like this: ```auto In [28]: import hail as hl ...: import pandas as pd ...: ...: mt = hl.utils.range_matrix_table(2, 2) ...: mt = mt.annotate_entries(x = (mt.row_idx + 1) * (mt.col_idx + 1)) ...: mt = mt.annotate_rows(row_field = hl.str('hello! ') + hl.str(mt.row_idx)) ...: ...: ...: sample_names = hl.delimit(hl.array([hl.str(mt[key_field]) for key_field in mt.col_key]), '_').collect() ...: t = mt.localize_entries('entries') ...: t = t.annotate(**{ ...: s + '_' + field: t.entries[i][field] ...: for i, s in enumerate(sample_names) ...: for field in mt.entry ...: }) ...: t = t.drop('entries') ...: df = pd.DataFrame(t.collect()) ...: df.set_index(list(mt.row_key), inplace=True) ...: df Out[28]: 0_x 1_x row_field row_idx 0 1 2 hello! 0 1 2 4 hello! 1 ``` ---
**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\ **Post date:** [December 14, 2021, 7:02pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/3 "2021-12-14T19:02:42Z")
Thank you so much for your response @danking! I am trying to export, but it seems to be taking a while. Is this an expensive operation? ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [December 14, 2021, 7:08pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/4 "2021-12-14T19:08:46Z")
How many rows and how many columns are in your matrix table? ---
**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\ **Post date:** [December 14, 2021, 7:17pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/5 "2021-12-14T19:17:00Z")
I believe there are at least a few hundred thousand rows… ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [December 14, 2021, 7:35pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/6 "2021-12-14T19:35:36Z")
If you have even a modest 1000 samples (columns), the absolute best case scenario is that your pandas data frame is about this big: ```nohighlight In [5]: 1000 * 500_000 * 8 / 1024 / 1024 / 1024 Out[5]: 3.725290298461914 ``` Nearly 3.7 GB. In reality it’s almost certainly bigger. How many columns do you have? And do you have any code in between: ```auto mt = hl.read_matrix_table(...) ``` and ```auto t = mt.localize_entries('entries') ``` ? If you share your whole script I can help you find slow parts, but, in general, you should avoid loading an entire genetics dataset into memory (e.g. into pandas). Most genetics datasets are require the use of streaming operations which Hail automatically uses. ---
**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\ **Post date:** [December 14, 2021, 8:33pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/7 "2021-12-14T20:33:17Z")
I just tried this: _new\_mt.entries().PBT\_GT.export(“gt.tsv”)_ and it somehow worked. The file was around 100+ MB although they are a few million lines. Does this file size make sense, or was it truncated? ---
**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\ **Post date:** [December 14, 2021, 8:45pm UTC](https://discuss.hail.is/t/exporting-data-from-matrixtable-into-tsv/2406/8 "2021-12-14T20:45:58Z")
I can’t say if that file size makes sense because I do not know how many columns/samples you have. In general, I have no reason to believe that command will truncate the output. If you’re exporting to text, I recommend using `gt.tsv.bgz` which will automatically compress the file and should be faster. `new_mt.entries().PBT_GT.export("gt.tsv")` will create one row per variant, per sample. So you’ll have n\_variants \* n\_samples rows. This is different from the VCF representation which has one row per variant.