Hail is 0.1! What does that mean?
0.1 is Hail's first stable release. "Stable release" can connote a lot of things, so here's what we mean:
- We will not be changing any interfaces in this version. Thanks to all of you for your patience as we broke your Hail scripts every week!
- We will support this version by fixing bugs and answering questions about its interfaces until it is deprecated. Deprecation will probably happen 3 months after the next stable version (0.2) is released.
- We will continue to add features (without changing existing interfaces!) to this version for now. This will probably stop in a month or so, as we shift to building features against changing development code for stable release in 0.2.
- We will continue automatically deploying new builds to the
gs://hail-common bucket. We may change how this is done in the next few days, though, so keep an eye on this forum!
Changes in 0.1
We've been saving up a pile of changes in the last month or so, in order not to break your scripts every few days. These changes all appear at once in the new 0.1 build. Here they are:
KeyTables: your new best friend for annotation and filtering
annotate_variants_table. These are essentially the functionality of the old
annotate_samples_keytable, with more. For example,
annotate_variants_table can be used to annotate variants by intervals, variants, or loci.
- Many annotation methods have disappeared! These can be implemented in terms of
annotate_variants_table. Some examples of removed methods:
- Added Keytable.import_fam, KeyTable.import_interval_list, and KeyTable.import_bed. These should be used with the new annotate methods.
HailContext's [import_table(https://hail.is/hail/hail.HailContext.html#hail.HailContext.import_table) (renamed from
import_keytable). The TextTableConfig object is gone, and all the parameters are now simply arguments to this method. Additionally, field names in tables imported with
no_header=True will be given names
fN" instead of
filter_samples_table. These take
KeyTables with key
Interval (for variants) or
String (for samples). Filtering from interval lists should go through this method. Don't worry -- we've kept the efficiency demonstrated in this blog post!
Pedigree as a first-class object
- We've added first-class Python objects for
Trio. Read fam files with the static
tdt methods now take a pedigree as an argument, instead of a fam file.
- Removed counts from LMM regression variant annotations. See the new schema in the docs here.
- Added dosage option to
grm method now returns a
KinshipMatrix object, which has new methods to export to all of the old GRM PLINK formats.
grm no longer takes a format argument.
- We used the term "dosage" in a very confusing (and incorrect) way.
g.dosage() in the expression language is now the expected number of alternate alleles given the genotype probabilities.
g.gp() is the linear-scaled genotype probabilities from which dosage is computed. This does not involve changes to the regression interfaces, though using
g.dosage() for a variant as a covariate will now be correct.
- We changed the
count method. It no longer takes a
genotypes boolean parameter or returns a dict: instead, it returns a tuple of (number of samples, number of variants).
- We added the
summarize method, which is an excellent way to get a broad sense of a dataset's contents.
Filter variants with Python objects
mendel_errors return key tables
mendel_errors now both return
concordance used to return the global statistics as a 2d list, and two variant datasets.
mendel_errors used to write four files. The docs explain these things clearly.
takeBy aggregator now takes elements from smallest to largest (used to be largest to smallest, but this was not clearly documented).
- HailContext.import_keytable => import_table
- HailContext.read_keytable => read_table
- VariantDataset.annotate_global_py => annotate_global
- VariantDataset.from_keytable => from_table
- VariantDataset.variants_keytable => variants_table
- VariantDataset.samples_keytable => samples_table
- VariantDataset.genotypes_keytable => genotypes_table
- VariantDataset.filter_variants_intervals => filter_intervals (some functionality moved to
- KeyTable.column_names => columns
- KeyTable.num_rows => count
Plus: Arguments are renamed in many functions ("code" is replaced with "expr" most places).
aggregate_intervals: this method is easy to implement in a few lines of KeyTable methods now.
annotate_global_table: rarely used functionality mostly replaced by KeyTable support. It is still possible to put tables into global annotations with
annotate_global_list: same as above.
IntervalTree. This object is totally obviated by the deep support for