# Updates **URL:** https://discuss.hail.is/c/updates/15.md [Latest](https://discuss.hail.is/latest.md) · [Categories](https://discuss.hail.is/categories.md) --- ## [About the Updates category](https://discuss.hail.is/t/about-the-updates-category/51)
**Author:** [@hail-team](https://discuss.hail.is/u/hail-team)\ **Replies:** 0
Hail's fast-growing log of changes. --- ## [Welcome to the Hail community!](https://discuss.hail.is/t/welcome-to-the-hail-community/8)
**Author:** [@system](https://discuss.hail.is/u/system)\ **Replies:** 0\ **Last updated:** [August 26, 2016, 7:39pm UTC](https://discuss.hail.is/t/welcome-to-the-hail-community/8 "2016-08-26T19:39:27Z")
Welcome to the discussion forum for Hail, an open-source, scalable framework for exploring and analyzing genetic data. The Hail team is based in the Neale Lab at the Stanley Center for Psychiatric Research of the Broad … --- ## [BlockMatrix.transpose Correctness Bug](https://discuss.hail.is/t/blockmatrix-transpose-correctness-bug/1420)
**Author:** [@johnc1231](https://discuss.hail.is/u/johnc1231)\ **Replies:** 0\ **Last updated:** [May 28, 2020, 5:31pm UTC](https://discuss.hail.is/t/blockmatrix-transpose-correctness-bug/1420 "2020-05-28T17:31:55Z")
Version 0.2.43 fixed a correctness bug in BlockMatrix.T (transpose). Previous versions have a bug where calling BlockMatrix.T on a matrix that is both sparse and non-symmetric matrix will result in incorrect results. Ple… --- ## [Possible incorrect linreg aggregator results in 0.2.29 - 0.2.37](https://discuss.hail.is/t/possible-incorrect-linreg-aggregator-results-in-0-2-29-0-2-37/1375)
**Author:** [@patrick-schultz](https://discuss.hail.is/u/patrick-schultz)\ **Replies:** 1\ **Last updated:** [April 17, 2020, 8:37pm UTC](https://discuss.hail.is/t/possible-incorrect-linreg-aggregator-results-in-0-2-29-0-2-37/1375 "2020-04-17T20:37:05Z")
We just fixed a bug in the linreg aggregator that has existed since release 0.2.29, and which will be fixed in 0.2.38. (It is fixed by this PR, which will merge into master shortly.) First off, any regressions containin… --- ## [Changes to VEP on hailctl, hail version 0.2.34, March 2020](https://discuss.hail.is/t/changes-to-vep-on-hailctl-hail-version-0-2-34-march-2020/1332)
**Author:** [@johnc1231](https://discuss.hail.is/u/johnc1231)\ **Replies:** 0\ **Last updated:** [March 12, 2020, 7:23pm UTC](https://discuss.hail.is/t/changes-to-vep-on-hailctl-hail-version-0-2-34-march-2020/1332 "2020-03-12T19:23:51Z")
Hail 0.2.34 was just released. This makes some changes to how VEP works in Google Cloud Dataproc. Specifically, the location of the VEP data stored in google cloud has moved into a set of requester-pays buckets, which Ha… --- ## [Performance regression in version 0.2.18](https://discuss.hail.is/t/performance-regression-in-version-0-2-18/1026)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [August 1, 2019, 7:11pm UTC](https://discuss.hail.is/t/performance-regression-in-version-0-2-18/1026 "2019-08-01T19:11:42Z")
We are in the unfortunate situation of finding and fixing a severe performance regression in back-to-back releases of Hail. In 0.2.19, we fixed a problem that only existed in 0.2.18 and led to extremely slow Table/Matrix… --- ## [Performance regression in versions 0.2.15-0.2.17](https://discuss.hail.is/t/performance-regression-in-versions-0-2-15-0-2-17/1015)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [July 12, 2019, 8:42pm UTC](https://discuss.hail.is/t/performance-regression-in-versions-0-2-15-0-2-17/1015 "2019-07-12T20:42:14Z")
In version 0.2.18, We resolved a severe performance regression that has existed since 0.2.15. The issue was related to code generation for constant strings and literals, and could cause slowdowns of 1-3 orders of magnitu… --- ## [New command-line utility: hailctl](https://discuss.hail.is/t/new-command-line-utility-hailctl/981)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [June 11, 2019, 10:11am UTC](https://discuss.hail.is/t/new-command-line-utility-hailctl/981 "2019-06-11T10:11:21Z")
hailctl As of 0.2.15, Hail comes bundled with the new command-line utility, hailctl. Short for “Hail control”, this tool is designed to be the home for functionality that helps manage Hail deployments, and more. You can… --- ## [New optimizer pass that extracts point queries and interval filters](https://discuss.hail.is/t/new-optimizer-pass-that-extracts-point-queries-and-interval-filters/979)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [June 7, 2019, 9:54pm UTC](https://discuss.hail.is/t/new-optimizer-pass-that-extracts-point-queries-and-interval-filters/979 "2019-06-07T21:54:28Z")
hl.filter\_intervals is one of the best ways to speed up pipelines that only need to look at a small fraction of your data, but the method is quite a pain to use compared to simpler methods like filter\_rows. As of 0.2.15… --- ## [Read from and write to hdfs/gs/s3 from Python](https://discuss.hail.is/t/read-from-and-write-to-hdfs-gs-s3-from-python/195)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 1\ **Last updated:** [April 6, 2019, 4:58pm UTC](https://discuss.hail.is/t/read-from-and-write-to-hdfs-gs-s3-from-python/195 "2019-04-06T16:58:35Z")
This is a feature that will make life much easier for many of you struggling to wrangle data from Python to Google buckets or Amazon S3. It comes in the form of three utility methods in the hail module: hadoop\_write h… --- ## [\[Feature\] Approximate quantiles, cdf and pdf plots](https://discuss.hail.is/t/feature-approximate-quantiles-cdf-and-pdf-plots/925)
**Author:** [@patrick-schultz](https://discuss.hail.is/u/patrick-schultz)\ **Replies:** 0\ **Last updated:** [April 3, 2019, 8:32pm UTC](https://discuss.hail.is/t/feature-approximate-quantiles-cdf-and-pdf-plots/925 "2019-04-03T20:32:23Z")
We have added a new aggregator called agg.approx\_cdf. Most users probably won’t need to use this directly, instead using one of the convenience methods wrapping it: The new aggregator agg.approx\_quantiles. The new plot… --- ## [Please help us find Hail-powered science](https://discuss.hail.is/t/please-help-us-find-hail-powered-science/905)
**Author:** [@jbloom](https://discuss.hail.is/u/jbloom)\ **Replies:** 0\ **Last updated:** [March 15, 2019, 2:22pm UTC](https://discuss.hail.is/t/please-help-us-find-hail-powered-science/905 "2019-03-15T14:22:19Z")
Hail team needs your help to create a list of preprints, publications, and projects that have used Hail in any way. Please submit links here: https://goo.gl/forms/DbqbppSOuqLImYPz1 Even better if you can explain how Ha… --- ## [Google Cloud Dataproc users: update cloudtools](https://discuss.hail.is/t/google-cloud-dataproc-users-update-cloudtools/901)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [March 11, 2019, 6:14pm UTC](https://discuss.hail.is/t/google-cloud-dataproc-users-update-cloudtools/901 "2019-03-11T18:14:39Z")
A Hail user recently discovered exorbitant ($600) charges from Stackdriver after using a small cluster of a few dozen machines over the course of a day. It is currently unclear whether this was simply a billing error or … --- ## [Hail 0.2.10 patch notes](https://discuss.hail.is/t/hail-0-2-10-patch-notes/866)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [February 15, 2019, 9:19pm UTC](https://discuss.hail.is/t/hail-0-2-10-patch-notes/866 "2019-02-15T21:19:46Z")
Looks like we haven’t posted since 0.2.6! 0.2.10 Released 2019-02-15 New features (#5272) Added a new ‘delimiter’ option to Table.export. (#5251) Add utility aliases to hl.plot for output\_notebook and show. (#5249) Ad… --- ## [Hail 0.2.6 patch notes](https://discuss.hail.is/t/hail-0-2-6-patch-notes/809)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [December 17, 2018, 2:07am UTC](https://discuss.hail.is/t/hail-0-2-6-patch-notes/809 "2018-12-17T02:07:16Z")
0.2.6 Released 2018-12-17 New features (#4962) Expanded comparison operators (==, !=, \<, \<=, \>, \>=) to support expressions of every type. (#4927) Expanded functionality of Table.order\_by to support ordering by arbitrar… --- ## [Hail 0.2.5 patch notes](https://discuss.hail.is/t/hail-0-2-5-patch-notes/798)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 5:16pm UTC](https://discuss.hail.is/t/hail-0-2-5-patch-notes/798 "2018-12-07T17:16:07Z")
0.2.5 Released 2018-12-07 New features (#4845) The or\_error method in hl.case and hl.switch statements now takes a string expression rather than a string literal, allowing more informative messages for errors and asser… --- ## [Announcing Hail weekly releases and patch notes](https://discuss.hail.is/t/announcing-hail-weekly-releases-and-patch-notes/797)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [December 7, 2018, 5:11pm UTC](https://discuss.hail.is/t/announcing-hail-weekly-releases-and-patch-notes/797 "2018-12-07T17:11:46Z")
First, let me note that this will not affect anyone’s current Hail use strategy - we have not changed the per-commit deployment to Google Cloud, so cloudtools will continue to work as before. From now on, we plan to mak… --- ## [NativeLib compilation now requires lz4](https://discuss.hail.is/t/nativelib-compilation-now-requires-lz4/732)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [October 30, 2018, 10:23am UTC](https://discuss.hail.is/t/nativelib-compilation-now-requires-lz4/732 "2018-10-30T10:23:50Z")
If you are compiling from source, you may be getting an error like: …/resources/include/hail/Encoder.h:3:17: fatal error: lz4.h: No such file or directory If so, then you need to install this dependency using a package… --- ## [\[Feature\] Chained linear regression](https://discuss.hail.is/t/feature-chained-linear-regression/726)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [October 26, 2018, 1:43am UTC](https://discuss.hail.is/t/feature-chained-linear-regression/726 "2018-10-26T01:43:06Z")
We’ve added a new capability to linear\_regression\_rows: the ability to run multiple regressions with different missingness patterns with one call to the function. The old (and preserved!) behavior is that passing a list… --- ## [\[Feature\] pass through fields to regression methods](https://discuss.hail.is/t/feature-pass-through-fields-to-regression-methods/725)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [October 26, 2018, 1:32am UTC](https://discuss.hail.is/t/feature-pass-through-fields-to-regression-methods/725 "2018-10-26T01:32:46Z")
We’ve added a new feature to the following methods: linear\_regression\_rows logistic\_regression\_rows poisson\_regression\_rows linear\_mixed\_regression\_rows These methods now all have a new argument, pass\_through. This is… --- ## [\[Breaking Change\] Hail 0.2 removed KinshipMatrix, linear\_mixed\_regression](https://discuss.hail.is/t/breaking-change-hail-0-2-removed-kinshipmatrix-linear-mixed-regression/578)
**Author:** [@jbloom](https://discuss.hail.is/u/jbloom)\ **Replies:** 17\ **Last updated:** [October 24, 2018, 6:04pm UTC](https://discuss.hail.is/t/breaking-change-hail-0-2-removed-kinshipmatrix-linear-mixed-regression/578 "2018-10-24T18:04:27Z")
(Update: Hail 0.2 has a function linear\_mixed\_regression\_rows which most closely implements the old linear\_mixed\_regression function) The KinshipMatrix class is gone. Instead, realized\_relationship\_matrix and genetic\_re… --- ## [Hail workshop on Oct 31, 2018 at Harvard School of Public Health](https://discuss.hail.is/t/hail-workshop-on-oct-31-2018-at-harvard-school-of-public-health/707)
**Author:** [@jbloom](https://discuss.hail.is/u/jbloom)\ **Replies:** 0\ **Last updated:** [October 22, 2018, 9:56pm UTC](https://discuss.hail.is/t/hail-workshop-on-oct-31-2018-at-harvard-school-of-public-health/707 "2018-10-22T21:56:57Z")
We’ll be running a spooky, introductory Hail workshop at 2pm on October 31 at the Harvard School of Public Health. More information and registration here. --- ## [Renamed filter\_variants\_all, filter\_samples\_all](https://discuss.hail.is/t/renamed-filter-variants-all-filter-samples-all/126)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [February 18, 2017, 11:47am UTC](https://discuss.hail.is/t/renamed-filter-variants-all-filter-samples-all/126 "2017-02-18T11:47:10Z")
Renamed filter\_variants\_all and filter\_samples\_all to drop\_variants and drop\_samples. --- ## [Faster Eigendecomposition (and maybe other Linear Algebra) with MKL](https://discuss.hail.is/t/faster-eigendecomposition-and-maybe-other-linear-algebra-with-mkl/266)
**Author:** [@johnc1231](https://discuss.hail.is/u/johnc1231)\ **Replies:** 0\ **Last updated:** [August 18, 2017, 8:50pm UTC](https://discuss.hail.is/t/faster-eigendecomposition-and-maybe-other-linear-algebra-with-mkl/266 "2017-08-18T20:50:54Z")
Just wanted to make a short post to alert people to availability of a potentially helpful init script. gs://hail-common/mkl/mkl-init.sh will install Intel’s MKL library on the your cluster, increasing the speed of linear… --- ## [Hail 0.2 seminar and workshop: April 10, Cologne](https://discuss.hail.is/t/hail-0-2-seminar-and-workshop-april-10-cologne/444)
**Author:** [@tpoterba](https://discuss.hail.is/u/tpoterba)\ **Replies:** 0\ **Last updated:** [March 27, 2018, 12:52pm UTC](https://discuss.hail.is/t/hail-0-2-seminar-and-workshop-april-10-cologne/444 "2018-03-27T12:52:34Z")
We’ll be running a 4-hour workshop on April 10 at the University of Cologne. If you’re nearby and would like to attend, see the link below for \[free\] registration! http://portal.ccg.uni-koeln.de/ccg/index.php?id=6 --- ## [\[Breaking change\] Hail 0.2 sign function](https://discuss.hail.is/t/breaking-change-hail-0-2-sign-function/549)
**Author:** [@jbloom](https://discuss.hail.is/u/jbloom)\ **Replies:** 0\ **Last updated:** [June 20, 2018, 10:09pm UTC](https://discuss.hail.is/t/breaking-change-hail-0-2-sign-function/549 "2018-06-20T22:09:27Z")
The signum method in 0.1 is called sign in 0.2 and, like NumPy, preserves the numeric type (technically, this is not a breaking change because we’d neglected to expose signum in 0.2!). --- ## [\[Breaking Change\] Hail 0.2 import\_bgen should be passed a min\_partitions parameter](https://discuss.hail.is/t/breaking-change-hail-0-2-import-bgen-should-be-passed-a-min-partitions-parameter/566)
**Author:** [@danking](https://discuss.hail.is/u/danking)\ **Replies:** 0\ **Last updated:** [July 12, 2018, 12:46am UTC](https://discuss.hail.is/t/breaking-change-hail-0-2-import-bgen-should-be-passed-a-min-partitions-parameter/566 "2018-07-12T00:46:28Z")
Previously, some Spark heuristics chose a reasonable number of partitions for import\_bgen. This logic has been removed while we improve bgen performance (we may add the logic back in before 0.2 release). In the meantime,… --- ## [\[Breaking Change\] split\_multi](https://discuss.hail.is/t/breaking-change-split-multi/601)
**Author:** [@wang](https://discuss.hail.is/u/wang)\ **Replies:** 0\ **Last updated:** [August 9, 2018, 6:04pm UTC](https://discuss.hail.is/t/breaking-change-split-multi/601 "2018-08-09T18:04:54Z")
the (non-hts) split multi interface has changed as of https://github.com/hail-is/hail/pull/4076. https://hail.is/docs/devel/methods/genetics.html#hail.methods.split\_multi split\_multi\_hts behavior should remain the same… --- ## [\[Breaking Change\] Random functions in Hail](https://discuss.hail.is/t/breaking-change-random-functions-in-hail/602)
**Author:** [@wang](https://discuss.hail.is/u/wang)\ **Replies:** 0\ **Last updated:** [August 10, 2018, 7:26pm UTC](https://discuss.hail.is/t/breaking-change-random-functions-in-hail/602 "2018-08-10T19:26:34Z")
The semantics of making multiple calls to the same random function have changed a little bit. You can read about the change here: https://hail.is/docs/devel/functions/random.html --- ## [\[Breaking Change\] VEP configuration format has changed](https://discuss.hail.is/t/breaking-change-vep-configuration-format-has-changed/604)
**Author:** [@cseed](https://discuss.hail.is/u/cseed)\ **Replies:** 0\ **Last updated:** [August 11, 2018, 9:56pm UTC](https://discuss.hail.is/t/breaking-change-vep-configuration-format-has-changed/604 "2018-08-11T21:56:06Z")
The VEP configuration format has changed to be more flexible. The new format is documented with the vep command. On Google Cloud Platform, cloudtools --vep option currently installed VEP v85 database files for referenc… [Next page](https://discuss.hail.is/c/updates/15.md?page=1)