# \[Breaking Change\] filter\_samples\_list now takes a list **URL:** \ **Category:** Updates\ **Created:** [April 7, 2017, 11:26pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185 "2017-04-07T23:26:59Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [April 7, 2017, 11:26pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/1 "2017-04-07T23:26:59Z")
As title. See the new docs: [https://hail.is/hail/hail.VariantDataset.html#hail.VariantDataset.filter\_samples\_list](https://hail.is/hail/hail.VariantDataset.html#hail.VariantDataset.filter_samples_list) ---
**Author:** ![Andrea\_Ganna](https://avatars.discourse-cdn.com/v4/letter/a/ecc23a/32.png) [@Andrea\_Ganna](https://discuss.hail.is/u/Andrea_Ganna)\ **Post date:** [April 10, 2017, 6:52pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/2 "2017-04-10T18:52:02Z")
Would be nice to add in the help an example regarding how to read a file, pass it to keytable, transform it in list and finally filter using `filter_samples_list` For example: ```auto sdrop=hc.import_keytable(stroot + 'samples_to_drop.txt', config=hail.TextTableConfig(noheader=True)).key_by('_0') sdropL = [item._0 for item in sdrop.collect()] vds.filter_samples_list(sdropL, keep=False) ``` ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [April 10, 2017, 8:37pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/3 "2017-04-10T20:37:58Z")
Why go through keytable? ```python samples = [] with open(filename, 'r') as f: for line in f: samples.append(line.strip()) vds = vds.filter_samples_list(samples) ``` ---
**Author:** ![Ruth\_Mitchell](https://avatars.discourse-cdn.com/v4/letter/r/ecae2f/32.png) [@Ruth\_Mitchell](https://discuss.hail.is/u/Ruth_Mitchell)\ **Post date:** [May 17, 2018, 3:06pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/4 "2018-05-17T15:06:57Z")
Has this been put into version 0.2? Or is this just a case of filtering columns as a bgen file now imports as a MatrixTable? In which case I can’t find how to filter columns based on a text file/list. Also, the same question holds for filtering variants. filter\_samples\_list() and filter\_variants\_table() were the two methods I was really hoping to use in Hail! ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [May 17, 2018, 3:17pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/5 "2018-05-17T15:17:47Z")
The easiest way to do both of these is going to be by importing to a table. Suppose I have a file that looks like this: ```auto NA12878 NA12891 NA12892 ... ``` And a file that looks like this: ```auto Variant 1:1:A:T 1:5:C:CC ... ``` Then the easiest way to keep these samples and these sites is going to be: ```python sample_table = hl.import_table(sample_file, no_header=True, key='f0') variant_table = hl.import_table(variant_file) # parse the chr:pos:ref:alt to locus / alleles fields variant_table = variant_table.key_by(**hl.parse_variant(variant_table['Variant'])) # filter to samples in the table mt = mt.filter_cols(hl.is_defined(sample_table[mt.col_key])) # filter to variants in the table mt = mt.filter_rows(hl.is_defined(variant_table[mt.row_key])) ``` ---
**Author:** ![Ruth\_Mitchell](https://avatars.discourse-cdn.com/v4/letter/r/ecae2f/32.png) [@Ruth\_Mitchell](https://discuss.hail.is/u/Ruth_Mitchell)\ **Post date:** [May 17, 2018, 4:12pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/6 "2018-05-17T16:12:23Z")
ok, yep, that makes sense. My variants are a list of rsids and I’m running into the error: TypeError: key\_by() got an unexpected keyword argument ‘locus’ I’ve tried changing mt key to ‘rsid’ so that they match but this shuts down the SparkContext! (Error summary: SparkException: Job 3 cancelled because SparkContext was shut down) With the samples, when I tried to filter them, I get a Py4JError (a different one every time I try!) Thanks for your help Tim! ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [May 17, 2018, 4:13pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/7 "2018-05-17T16:13:39Z")
oh, oops – what version are you using? The key\_by interface was changed about a week ago. What py4j error? that sounds bad. ---
**Author:** ![Ruth\_Mitchell](https://avatars.discourse-cdn.com/v4/letter/r/ecae2f/32.png) [@Ruth\_Mitchell](https://discuss.hail.is/u/Ruth_Mitchell)\ **Post date:** [May 17, 2018, 4:23pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/8 "2018-05-17T16:23:14Z")
Ah ok, I’m using a version from the 8th May - I can’t find the actually number. I’ll get repulled. I’ve had several… Py4JError: An error occurred while calling o95.selectCols Py4JError: An error occurred while calling o39.annotateColsTable Py4JError: An error occurred while calling o112.selectCols They all seem to be from here: ./Hail/spark-2.2.0-bin-hadoop2.7/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py I was getting similar ones this morning as I was trying to run through the tutorials (I decided to leave them and see if I could do what I actually wanted to do in Hail). ---
**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Post date:** [May 17, 2018, 4:34pm UTC](https://discuss.hail.is/t/breaking-change-filter-samples-list-now-takes-a-list/185/9 "2018-05-17T16:34:05Z")
can you make a new discuss post with the issues you’re seeing? Possibly it’s due to a mismatch between the Python and the Jar (if it’s something about “no method with signature blah blah blah”)