# Annotating a variants in a hail matrix with de novo **URL:** \ **Category:** Hail Query & hailctl\ **Created:** [November 6, 2020, 8:40pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776 "2020-11-06T20:40:36Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 8:40pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/1 "2020-11-06T20:40:36Z")
When I run the de\_novo calling using hail de\_novo(), I obtain a hail table. I was wondering if there was a straight forward way to annotate the initial hail matrix entries with whether or not the variant was de novo or not from the hail table? Thank you! ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [November 6, 2020, 9:05pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/2 "2020-11-06T21:05:24Z")
I think you can do something like: ```auto de_novo_table = hl.de_novo(mt, ...) mt = mt.annotate_entries(was_called_de_novo = hl.is_defined(de_novo_table[mt.row_key, mt.col_key])) ``` ---
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 9:14pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/3 "2020-11-06T21:14:38Z")
When I try that approach, I get the following error message: ExpressionException: Key type mismatch: cannot index table with given expressions: Table key: locus, array, str Index Expressions: struct{locus: locus, alleles: array}, struct{s: str} ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [November 6, 2020, 9:16pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/4 "2020-11-06T21:16:22Z")
oh, oops! should be `[mt.locus, mt.alleles, mt.s]` in the brackets, I think. Might need this above as well: ```auto de_novo_table = de_novo_table.key_by('locus', 'alleles', 'id') ``` ---
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 9:37pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/5 "2020-11-06T21:37:09Z")
I still seem to have the same error: ExpressionException: Key type mismatch: cannot index table with given expressions: Table key: locus, array, str Index Expressions: struct{locus: locus, alleles: array}, struct{s: str}, str ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [November 6, 2020, 9:45pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/6 "2020-11-06T21:45:32Z")
> [@tpoterba](#): > > [mt.locus, mt.alleles, mt.s] are you indexing with these instead of `[mt.row_key, mt.col_key]` now? I’d expect that error if you aren’t. ---
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 9:49pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/7 "2020-11-06T21:49:51Z")
oh haha yeah, I was doing the opposite. But now I have the following error: * * * NotImplementedError Traceback (most recent call last) in ----\> 1 asc\_mt = asc\_mt.annotate\_entries(is\_denovo = hl.is\_defined(de\_novo\_HC[asc\_mt.locus, asc\_mt.alleles, asc\_mt.s])) /usr/local/lib/python3.7/dist-packages/hail/table.py in **getitem** (self, item) 372 373 try: –\> 374 return self.index(\*wrap\_to\_tuple(item)) 375 except TypeError as e: 376 raise TypeError(“Table. **getitem** : invalid index argument(s)\n” /usr/local/lib/python3.7/dist-packages/hail/table.py in index(self, all\_matches, \*exprs) 1547 “”" 1548 try: -\> 1549 return self.\_index(\*exprs, all\_matches=all\_matches) 1550 except TableIndexKeyError as err: 1551 raise ExpressionException(f"Key type mismatch: cannot index table with given expressions:\n" /usr/local/lib/python3.7/dist-packages/hail/table.py in \_index(self, all\_matches, \*exprs) 1677 # match on indices to determine join type 1678 if indices == src.\_entry\_indices: -\> 1679 raise NotImplementedError(‘entry-based matrix joins’) 1680 elif indices == src.\_row\_indices: 1681 is\_subset\_row\_key = len(exprs) \<= len(src.row\_key) and all( NotImplementedError: entry-based matrix joins ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [November 6, 2020, 9:54pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/8 "2020-11-06T21:54:38Z")
> [@emauryg](#): > > NotImplementedError: entry-based matrix joins oh, that’s super annoying – could have sworn we had this. Anyway, here’s a workaround; ```python de_novo_table = hl.de_novo(mt, ...) de_novo_mt = de_novo_table.to_matrix_table(row_key=['locus', 'alleles'], col_key=['id']) mt = mt.annotate_entries(was_called_de_novo = hl.is_defined(de_novo_mt[mt.row_key, mt.col_key])) ``` ---
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 10:00pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/9 "2020-11-06T22:00:35Z")
Thank you that seems to work! with the following warning message: 2020-11-06 21:58:06 Hail: WARN: cols(): Resulting column table is sorted by ‘col\_key’. To preserve matrix table column order, first unkey columns with ‘key\_cols\_by()’ ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [November 6, 2020, 10:02pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/10 "2020-11-06T22:02:14Z")
This is safe to ignore. It’s a message printed the first time you call `mt.cols()` because sometimes people are surprised that the sample order in the matrix table is not preserved in the columns table (currently, Hail guarantees that keyed tables are sorted, while columns of a matrix table are NOT sorted by key). The internal implementation of to\_matrix\_table calls `.cols()` internally. ---
**Author:** ![emauryg](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/emauryg/32/502_2.png) [@emauryg](https://discuss.hail.is/u/emauryg)\ **Post date:** [November 6, 2020, 10:05pm UTC](https://discuss.hail.is/t/annotating-a-variants-in-a-hail-matrix-with-de-novo/1776/11 "2020-11-06T22:05:15Z")
Great, thank you!