# Extracting entry fields into a separate MatrixTable

**URL:** <https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405>\
**Category:** Hail Query & hailctl\
**Created:** [December 13, 2021, 4:42pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405 "2021-12-13T16:42:26Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 13, 2021, 4:42pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/1 "2021-12-13T16:42:26Z")

</div>

Hi all! I have three entry fields in a MatrixTable:  
_phased\_trio\_dataset.entries().mother\_entry.PBT\_GT_  
_phased\_trio\_dataset.entries().father\_entry.PBT\_GT_  
_phased\_trio\_dataset.entries().proband\_entry.PBT\_GT_  
They are all CallExpressions. Is there a way to save these entries (and the corresponding variants) into a separate MatrixTable? Thank you very much!

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [December 13, 2021, 4:48pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/2 "2021-12-13T16:48:12Z")

</div>

Yes!

```auto
new_mt = phased_trio_dataset.select_entries(
    mother_entry = phased_trio_dataset.mother_entry.select('PBT_GT'),
    father_entry = phased_trio_dataset.father_entry.select('PBT_GT'),
    proband_entry = phased_trio_dataset.proband_entry.select('PBT_GT')
)

```

* * *

Aside: `.entries()` is a very inefficient operation which converts from a 2-d matrix representation to a 1-d table representation. I don’t recommend using it!

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 13, 2021, 4:50pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/3 "2021-12-13T16:50:30Z")

</div>

> [@CuriousGeneticist](#):
>
> They are all CallExpressions. Is there a way to save these entries (and the corresponding variants) into a separate MatrixTable? Thank you very much!

What’s your end goal here? Why is it important that these are in a separate matrixtable? The MatrixTable containing these three fields can probably be used to do what you want.

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 13, 2021, 4:57pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/4 "2021-12-13T16:57:26Z")

</div>

Thank you so much for your response! I tried something similar myself and got the same result:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/4e52c075aac38a075059ce91bbb9127ff83c94ef.jpeg)  
I am not sure of what went wrong.

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 13, 2021, 4:58pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/5 "2021-12-13T16:58:23Z")

</div>

## This is the what I get with describe(): Page up

```
'proband_entry': struct {
    AD: array<int32>, 
    DP: int32, 
    GQ: int32, 
    GT: call, 
    MIN_DP: int32, 
    PGT: call, 
    PID: str, 
    PL: array<int32>, 
    PS: int32, 
    RGQ: int32, 
    SB: array<int32>, 
    PBT_GT: call
}
'father_entry': struct {
    AD: array<int32>, 
    DP: int32, 
    GQ: int32, 
    GT: call, 
    MIN_DP: int32, 
    PGT: call, 
    PID: str, 
    PL: array<int32>, 
    PS: int32, 
    RGQ: int32, 
    SB: array<int32>, 
    PBT_GT: call
}
'mother_entry': struct {
    AD: array<int32>, 
    DP: int32, 
    GQ: int32, 
    GT: call, 
    MIN_DP: int32, 
    PGT: call, 
    PID: str, 
    PL: array<int32>, 
    PS: int32, 
    RGQ: int32, 
    SB: array<int32>, 
    PBT_GT: call
}

```

---

<div class="post-metadata">

**Author:** ![danking](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/danking/32/43_2.png) [@danking](https://discuss.hail.is/u/danking)\
**Post date:** [December 13, 2021, 5:08pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/6 "2021-12-13T17:08:41Z")

</div>

Ah, my bad, I forgot to repeat the name of the mt when referring to the entry fields:

```auto
new_mt = phased_trio_dataset.select_entries(
    mother_entry = phased_trio_dataset.mother_entry.select('PBT_GT'),
    father_entry = phased_trio_dataset.father_entry.select('PBT_GT'),
    proband_entry = phased_trio_dataset.proband_entry.select('PBT_GT')
)

```

I’ve also updated my original post.

Aside: This is why I tend to use really short names for my matrix tables like `mt` or `ds`.

---

<div class="post-metadata">

**Author:** ![CuriousGeneticist](https://avatars.discourse-cdn.com/v4/letter/c/6f9a4e/32.png) [@CuriousGeneticist](https://discuss.hail.is/u/CuriousGeneticist)\
**Post date:** [December 14, 2021, 6:33pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/7 "2021-12-14T18:33:15Z")

</div>

Hi @danking, thank you so much for that! Apologies for a quick follow-up question, but I am trying to retain only the heterozygous phased SNPs in the mt with this:  
_new\_mt\_het = new\_mt.filter\_entries(hl.agg.any(new\_mt.entries().proband\_entry.PBT\_GT.is\_het()))_  
However, that gave me the error:

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/hail/original/1X/ad0e2668858e503b21bb15ca30113f49662eef9d.jpeg)  
May I know what went wrong? It worked for me before.

---

<div class="post-metadata">

**Author:** ![johnc1231](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/johnc1231/32/286_2.png) [@johnc1231](https://discuss.hail.is/u/johnc1231)\
**Post date:** [December 14, 2021, 6:42pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/8 "2021-12-14T18:42:34Z")

</div>

```auto
 new_mt_het = new_mt.filter_entries(hl.agg.any(new_mt.proband_entry.PBT_GT.is_het()))

```

I took out the `.entries()` after `new_mt`. That was the problem. Calling `entries` is saying “make me a new table based on this MatrixTable that has a row for each entry”. You were getting the `proband_entry` field of that new table, instead of the `MatrixTable`, so it told you were using a mix of expressions from two different data sources.

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [December 14, 2021, 6:44pm UTC](https://discuss.hail.is/t/extracting-entry-fields-into-a-separate-matrixtable/2405/9 "2021-12-14T18:44:22Z")

</div>

@CuriousGeneticist

if you haven’t, you should take a look at the visual cheat sheets, which can provide visualizations of what various table / matrixtable operations are doing:

[https://hail.is/docs/0.2/cheatsheets.html](https://hail.is/docs/0.2/cheatsheets.html)
