# Mergin MatrixTable raised strange row type error
**URL:** https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108
**Category:** Hail Query & hailctl
**Created:** [June 28, 2021, 10:02am UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108 "2021-06-28T10:02:02Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@carleshf](https://discuss.hail.is/u/carleshf)
#### Post date: [June 28, 2021, 10:02am UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/1 "2021-06-28T10:02:02Z")
I created two MatrixTable (sparse matrices: `old` and `new`) and I want to merge them. I typically use the experimental function `combine_gvcfs`. But I am now in a strange case resulting in an error:
```auto
TypeError: All input tables to multi_way_zip_join must have the same row type
```
I wen to the code of the ([`multi_way_zip_join`](https://hail.is/docs/0.2/_modules/hail/table.html)) in order to identify why this error is raised (and why I’ve never seen it before) and I tried to reproduce it.
- Test one
```auto
old.row.dtype == new.row.dtype, \
old.globals.dtype == new.globals.dtype, \
old.row_key.dtype == new.row_key.dtype, \
old.col_key.dtype == new.col_key.dtype
```
Returned:
```auto
(True, True, True, True)
```
- Test two:
```auto
tables = [old, new]
head = tables[0]
any(head.row.dtype != t.row.dtype for t in tables)
```
Returned:
```auto
False
```
I also went through [this](https://discuss.hail.is/t/hail-0-2-merge-two-matrixtable/551/12) but I don’t think it is the right thread, right?
Any idea?
* * *
- Running on Apache Spark version 3.1.1
- HAIL version 0.2.65
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 28, 2021, 11:51am UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/2 "2021-06-28T11:51:50Z")
Can you paste the stack trace to this error? I agree, this seems strange.
---
### Author:  [@carleshf](https://discuss.hail.is/u/carleshf)
#### Post date: [June 28, 2021, 11:54am UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/3 "2021-06-28T11:54:55Z")
Hi!
Of course! Here it is:
```auto
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in
1 from hail.experimental.vcf_combiner.vcf_combiner import combine_gvcfs
2
----> 3 comb = combine_gvcfs([old, new])
/usr/local/lib/python3.9/site-packages/decorator.py in fun(*args, **kw)
230 if not kwsyntax:
231 args, kw = fix(args, kw, sig)
--> 232 return caller(func, *(extras + args), **kw)
233 fun. __name__ = func. __name__
234 fun. __doc__ = func. __doc__
/usr/local/lib/python3.9/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
575 def wrapper(__original_func, *args, **kwargs):
576 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 577 return __original_func(*args_, **kwargs_)
578
579 return wrapper
/usr/local/lib/python3.9/site-packages/hail/experimental/vcf_combiner/vcf_combiner.py in combine_gvcfs(mts)
283 module provides no method of repartitioning data.
284 """
--> 285 ts = hl.Table.multi_way_zip_join([localize(mt) for mt in mts], 'data', 'g')
286 combined = combine(ts)
287 return unlocalize(combined)
/usr/local/lib/python3.9/site-packages/decorator.py in fun(*args, **kw)
230 if not kwsyntax:
231 args, kw = fix(args, kw, sig)
--> 232 return caller(func, *(extras + args), **kw)
233 fun. __name__ = func. __name__
234 fun. __doc__ = func. __doc__
/usr/local/lib/python3.9/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
575 def wrapper(__original_func, *args, **kwargs):
576 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 577 return __original_func(*args_, **kwargs_)
578
579 return wrapper
/usr/local/lib/python3.9/site-packages/hail/table.py in multi_way_zip_join(tables, data_field_name, global_field_name)
3487 raise TypeError('All input tables to multi_way_zip_join must have the same key type')
3488 if any(head.row.dtype != t.row.dtype for t in tables):
-> 3489 raise TypeError('All input tables to multi_way_zip_join must have the same row type')
3490 if any(head.globals.dtype != t.globals.dtype for t in tables):
3491 raise TypeError('All input tables to multi_way_zip_join must have the same global type')
TypeError: All input tables to multi_way_zip_join must have the same row type
```
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 28, 2021, 12:00pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/4 "2021-06-28T12:00:50Z")
We should fix the error message here to provide printouts of the types. For now, let’s try monkey patching:
```python
def monkeypatched_zip_join(tables, data_field_name, global_field_name):
head = tables[0]
if any(head.row.dtype != t.row.dtype for t in tables):
raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
hl.Table.multi_way_zip_join = monkeypatched_zip_join
```
Try that above your combine\_gvcfs call, it should give us more info.
---
### Author:  [@carleshf](https://discuss.hail.is/u/carleshf)
#### Post date: [June 28, 2021, 12:06pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/5 "2021-06-28T12:06:46Z")
Okay…
I think this goes in the same direction of my “test one”, which seems correct. Here the output:
```auto
TypeError: All input tables to multi_way_zip_join must have the same row type
struct{locus: locus, alleles: array, rsid: str, __entries: array, LAD: array, LGT: call, LPGT: call, LPL: array, MIN_DP: int32, PID: str, RGQ: int32, SB: array, gvcf_info: struct{BaseQRankSum: float64, DB: bool, ExcessHet: float64, InbreedingCoeff: float64, MLEAC: array, MLEAF: array, MQRankSum: float64, RAW_MQandDP: array, ReadPosRankSum: float64, MQ_DP: int32, VarDP: int32, QUALapprox: int32}}>}
struct{locus: locus, alleles: array, rsid: str, __entries: array, LGT: call, LAD: array, LPGT: call, LPL: array, RGQ: int32, END: int32, gvcf_info: struct{BaseQRankSum: float64, DB: bool, ExcessHet: float64, InbreedingCoeff: float64, MLEAC: array, MLEAF: array, MQRankSum: float64, RAW_MQandDP: array, ReadPosRankSum: float64, MQ_DP: int32, VarDP: int32, QUALapprox: int32}, DP: int32, GQ: int32, MIN_DP: int32, PID: str, SB: array}>}
```
Should I order in the same way the content of `gvcf_info`? To see:
- `old` structure:
```auto
old.describe()
----------------------------------------
Global fields:
None
----------------------------------------
Column fields:
's': str
----------------------------------------
Row fields:
'locus': locus
'alleles': array
'rsid': str
----------------------------------------
Entry fields:
'DP': int32
'END': int32
'GQ': int32
'LA': array
'LAD': array
'LGT': call
'LPGT': call
'LPL': array
'MIN_DP': int32
'PID': str
'RGQ': int32
'SB': array
'gvcf_info': struct {
BaseQRankSum: float64,
DB: bool,
ExcessHet: float64,
InbreedingCoeff: float64,
MLEAC: array,
MLEAF: array,
MQRankSum: float64,
RAW_MQandDP: array,
ReadPosRankSum: float64,
MQ_DP: int32,
VarDP: int32,
QUALapprox: int32
}
----------------------------------------
Column key: ['s']
Row key: ['locus']
----------------------------------------
```
- `new` structure:
```auto
new.describe()
----------------------------------------
Global fields:
None
----------------------------------------
Column fields:
's': str
----------------------------------------
Row fields:
'locus': locus
'alleles': array
'rsid': str
----------------------------------------
Entry fields:
'LA': array
'LGT': call
'LAD': array
'LPGT': call
'LPL': array
'RGQ': int32
'END': int32
'gvcf_info': struct {
BaseQRankSum: float64,
DB: bool,
ExcessHet: float64,
InbreedingCoeff: float64,
MLEAC: array,
MLEAF: array,
MQRankSum: float64,
RAW_MQandDP: array,
ReadPosRankSum: float64,
MQ_DP: int32,
VarDP: int32,
QUALapprox: int32
}
'DP': int32
'GQ': int32
'MIN_DP': int32
'PID': str
'SB': array
----------------------------------------
Column key: ['s']
Row key: ['locus']
----------------------------------------
```
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 28, 2021, 12:19pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/6 "2021-06-28T12:19:46Z")
Ah, the issue is out-of-order entry fields (these methods assume the same order).
You can unify, I think, by doing:
```python
new2 = new.select_entries(*old.entry)
```
which is an alias for
```python
new2 = new.select_entries('DP', 'END', 'GQ', ...)
```
---
### Author:  [@carleshf](https://discuss.hail.is/u/carleshf)
#### Post date: [June 28, 2021, 12:24pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/7 "2021-06-28T12:24:39Z")
So… Yes, it seems it solves the main issue although it raised a new one, related to the level of recursion. Here the stack trace:
```auto
---------------------------------------------------------------------------
RecursionError Traceback (most recent call last)
in
12 new3 = new.select_entries(*old.entry)
13
---> 14 comb = combine_gvcfs([old, new2])
/usr/local/lib/python3.9/site-packages/decorator.py in fun(*args, **kw)
230 if not kwsyntax:
231 args, kw = fix(args, kw, sig)
--> 232 return caller(func, *(extras + args), **kw)
233 fun. __name__ = func. __name__
234 fun. __doc__ = func. __doc__
/usr/local/lib/python3.9/site-packages/hail/typecheck/check.py in wrapper(__original_func, *args, **kwargs)
575 def wrapper(__original_func, *args, **kwargs):
576 args_, kwargs_ = check_all(__original_func, args, kwargs, checkers, is_method=is_method)
--> 577 return __original_func(*args_, **kwargs_)
578
579 return wrapper
/usr/local/lib/python3.9/site-packages/hail/experimental/vcf_combiner/vcf_combiner.py in combine_gvcfs(mts)
283 module provides no method of repartitioning data.
284 """
--> 285 ts = hl.Table.multi_way_zip_join([localize(mt) for mt in mts], 'data', 'g')
286 combined = combine(ts)
287 return unlocalize(combined)
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
... last 5 frames repeated, from the frame below ...
in monkeypatched_zip_join(tables, data_field_name, global_field_name)
6 if any(head.row.dtype != t.row.dtype for t in tables):
7 raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
----> 8 return hl.Table.multi_way_zip_join(tables, data_field_name, global_field_name)
9
10 hl.Table.multi_way_zip_join = monkeypatched_zip_join
RecursionError: maximum recursion depth exceeded while calling a Python object
```
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 28, 2021, 12:26pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/8 "2021-06-28T12:26:50Z")
oh, oops, I messed up the monkey patch – should have looked something like:
```auto
old_zip_join hl.Table.multi_way_zip_join
def monkeypatched_zip_join(tables, data_field_name, global_field_name):
head = tables[0]
if any(head.row.dtype != t.row.dtype for t in tables):
raise TypeError(f'All input tables to multi_way_zip_join must have the same row type\n ' + '\n '.join(str(t.row.dtype) for t in tables))
return old_zip_join(tables, data_field_name, global_field_name)
hl.Table.multi_way_zip_join = monkeypatched_zip_join
```
---
### Author:  [@tpoterba](https://discuss.hail.is/u/tpoterba)
#### Post date: [June 28, 2021, 12:26pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/9 "2021-06-28T12:26:58Z")
you can take all that out though if the types are now working
---
### Author:  [@carleshf](https://discuss.hail.is/u/carleshf)
#### Post date: [June 28, 2021, 12:34pm UTC](https://discuss.hail.is/t/mergin-matrixtable-raised-strange-row-type-error/2108/10 "2021-06-28T12:34:07Z")
Perfect! The issue was the order of the fields.
Thanks!