# Help \[0.1\] **URL:** https://discuss.hail.is/c/help-01/6.md?page=5 [Latest](https://discuss.hail.is/latest.md) · [Categories](https://discuss.hail.is/categories.md) **Page:** 6 --- ## [Hail installation in Cloudera](https://discuss.hail.is/t/hail-installation-in-cloudera/282)
**Author:** [@fang](https://discuss.hail.is/u/fang)\ **Replies:** 1\ **Last updated:** [September 11, 2017, 8:52pm UTC](https://discuss.hail.is/t/hail-installation-in-cloudera/282 "2017-09-11T20:52:00Z")
We are trying to install Hail in our Cloudera/Hadoop system. We plan to install Hail in one of gateway node in our system. We met several issues during our installation as follows: When we tried to get source code from… --- ## [Got error when writing vds with parquet\_genotypes parameter](https://discuss.hail.is/t/got-error-when-writing-vds-with-parquet-genotypes-parameter/276)
**Author:** [@jerryliu2005](https://discuss.hail.is/u/jerryliu2005)\ **Replies:** 2\ **Last updated:** [September 8, 2017, 12:31pm UTC](https://discuss.hail.is/t/got-error-when-writing-vds-with-parquet-genotypes-parameter/276 "2017-09-08T12:31:06Z")
Hi, I can write vds using the “parquet\_genotypes” parameter using hail 0.1 but not in the dev version. Here’s the error I got: vds=hc.import\_vcf(“1kg.vcf.bgz”) vds.write(“1kg\_pq.vds”, parquet\_genotypes=True) TypeErro… --- ## [Importing non GATK VCF into hail](https://discuss.hail.is/t/importing-non-gatk-vcf-into-hail/274)
**Author:** [@jobaba](https://discuss.hail.is/u/jobaba)\ **Replies:** 4\ **Last updated:** [September 6, 2017, 8:37pm UTC](https://discuss.hail.is/t/importing-non-gatk-vcf-into-hail/274 "2017-09-06T20:37:32Z")
I was wondering if it is possible to import non-GATK VCFs into Hail. Our VCF files were generated by Illumina Services using Isaac variant caller which doesn’t have the PL values. We were wondering if we could change the… --- ## [Out of Space when writting VDS](https://discuss.hail.is/t/out-of-space-when-writting-vds/271)
**Author:** [@jiaoweitang](https://discuss.hail.is/u/jiaoweitang)\ **Replies:** 6\ **Last updated:** [September 5, 2017, 10:15am UTC](https://discuss.hail.is/t/out-of-space-when-writting-vds/271 "2017-09-05T10:15:36Z")
Hi All, I have a vcf.bgz file and it is around 300 G. I would like to convert it to VDS, However, Hail generated very big files during writing. There are some examples: hdfs dfs -du -s -h /applications/genomic/data/dat… --- ## [Not able to write to vds](https://discuss.hail.is/t/not-able-to-write-to-vds/269)
**Author:** [@jobaba](https://discuss.hail.is/u/jobaba)\ **Replies:** 9\ **Last updated:** [September 1, 2017, 9:10pm UTC](https://discuss.hail.is/t/not-able-to-write-to-vds/269 "2017-09-01T21:10:56Z")
Hi, I am able to import vcf by vds = hc.import\_vcf(‘1kg.vcf.bgz’) and I am able to get the summarize report by vds.summarize().report() however, i got the following error when I tried to write vds. Could you help? T… --- ## [Initialize array with dynamic length?](https://discuss.hail.is/t/initialize-array-with-dynamic-length/270)
**Author:** [@mpinese](https://discuss.hail.is/u/mpinese)\ **Replies:** 2\ **Last updated:** [September 1, 2017, 11:18am UTC](https://discuss.hail.is/t/initialize-array-with-dynamic-length/270 "2017-09-01T11:18:30Z")
Is there any way to create an array with a dynamic length, similar to Python’s \[0\] \* length construct? I’m looking for something to plug into the following: if (g.isCalled()) g.oneHotAlleles(v) else \[0\]\*v.nAlleles() I… --- ## ["Insufficient Memory" + performance issues on Google Cloud](https://discuss.hail.is/t/insufficient-memory-performance-issues-on-google-cloud/259)
**Author:** [@DylanVrana](https://discuss.hail.is/u/DylanVrana)\ **Replies:** 4\ **Last updated:** [July 26, 2017, 8:23pm UTC](https://discuss.hail.is/t/insufficient-memory-performance-issues-on-google-cloud/259 "2017-07-26T20:23:47Z")
I’m using Hail to perform a GWAS for some expression QTLs on Google Cloud. The code is 1) performing a series of regressions w/ linreg\_multipheno, 2) after each block of phenotypes, joining the variant\_table output to … --- ## [Working with VDS files using the sparklyr R package + plans for R API?](https://discuss.hail.is/t/working-with-vds-files-using-the-sparklyr-r-package-plans-for-r-api/193)
**Author:** [@jhuyghe](https://discuss.hail.is/u/jhuyghe)\ **Replies:** 2\ **Last updated:** [July 22, 2017, 3:52pm UTC](https://discuss.hail.is/t/working-with-vds-files-using-the-sparklyr-r-package-plans-for-r-api/193 "2017-07-22T15:52:27Z")
Hello, Since a VDS file is based on the Parquet file format, I was wondering whether anyone has experience working with VDS files within an R environment using the sparklyr package. Also, are there ongoing or planned e… --- ## [Hail unable to see sample vcf files](https://discuss.hail.is/t/hail-unable-to-see-sample-vcf-files/174)
**Author:** [@mweiner](https://discuss.hail.is/u/mweiner)\ **Replies:** 13\ **Last updated:** [July 18, 2017, 7:47pm UTC](https://discuss.hail.is/t/hail-unable-to-see-sample-vcf-files/174 "2017-07-18T19:47:08Z")
In running through the ‘Getting Started’ i am trying to import the included sample.vcf into Hail’s .vds format, run: but when i run the command in the python interperter i get the error: \[cloudera@quickstart hail\]$ pyth… --- ## [Java error when loading VCF](https://discuss.hail.is/t/java-error-when-loading-vcf/251)
**Author:** [@klong](https://discuss.hail.is/u/klong)\ **Replies:** 8\ **Last updated:** [July 18, 2017, 6:12pm UTC](https://discuss.hail.is/t/java-error-when-loading-vcf/251 "2017-07-18T18:12:25Z")
I am trying to load a 1000 genome VCF and I get this error. Seems like it is trying to use Spark1? Maybe? Just guessing. Hail version: 0.1-f57188f Error summary: ClassCastException: cannot assign instance of scala.coll… --- ## [Invalid AD field](https://discuss.hail.is/t/invalid-ad-field/254)
**Author:** [@jiaoweitang](https://discuss.hail.is/u/jiaoweitang)\ **Replies:** 1\ **Last updated:** [July 18, 2017, 3:47pm UTC](https://discuss.hail.is/t/invalid-ad-field/254 "2017-07-18T15:47:50Z")
Hi All, I was trying to import vcf.bgz with HailContext and got errors about "invalid AD field". Here is an example: at java.lang.Thread.run(Thread.java:745)is.hail.utils.HailException: xxxx.vcf.bgz: invalid AD field \`… --- ## [Partition the VDS before querying it](https://discuss.hail.is/t/partition-the-vds-before-querying-it/248)
**Author:** [@pretty-speeches](https://discuss.hail.is/u/pretty-speeches)\ **Replies:** 6\ **Last updated:** [July 8, 2017, 1:13pm UTC](https://discuss.hail.is/t/partition-the-vds-before-querying-it/248 "2017-07-08T13:13:05Z")
Hi, I'm trying to run an analysis on my laptop, and I want to partition the VDS and perform an analysis on each partition separately. I am looking for a method similar to VDS.sample\_variants(frac), but one that will all… --- ## [AttributeError: 'VariantDataset' object has no attribute 'summarize'](https://discuss.hail.is/t/attributeerror-variantdataset-object-has-no-attribute-summarize/246)
**Author:** [@pretty-speeches](https://discuss.hail.is/u/pretty-speeches)\ **Replies:** 3\ **Last updated:** [July 5, 2017, 12:36pm UTC](https://discuss.hail.is/t/attributeerror-variantdataset-object-has-no-attribute-summarize/246 "2017-07-05T12:36:06Z")
I got the error in the title and solved it, if anyone has the same problem - my spark versions didn't match. I changed the following line: $ ./gradlew -Dspark.version=2.0.2 shadowJar to: $ ./gradlew -Dspark.version=2.1.… --- ## [Command to block gzip a vcf file](https://discuss.hail.is/t/command-to-block-gzip-a-vcf-file/243)
**Author:** [@JAK](https://discuss.hail.is/u/JAK)\ **Replies:** 2\ **Last updated:** [June 30, 2017, 8:23pm UTC](https://discuss.hail.is/t/command-to-block-gzip-a-vcf-file/243 "2017-06-30T20:23:47Z")
What is the command to block gzip a vcf file? --- ## [Updating info field after allele filter for non-split VDS](https://discuss.hail.is/t/updating-info-field-after-allele-filter-for-non-split-vds/239)
**Author:** [@xiaoli](https://discuss.hail.is/u/xiaoli)\ **Replies:** 0\ **Last updated:** [June 21, 2017, 3:04pm UTC](https://discuss.hail.is/t/updating-info-field-after-allele-filter-for-non-split-vds/239 "2017-06-21T15:04:34Z")
Info fields are out of dated once allele specific filters are applied. It will also happen when subsetting samples. In most of occasions, where VDS is split into biallelics, you can directly use variant\_qc to update AF, … --- ## [Importing VCFs where GT is 0/2 but AD has only 2 entries](https://discuss.hail.is/t/importing-vcfs-where-gt-is-0-2-but-ad-has-only-2-entries/234)
**Author:** [@wmcfadden](https://discuss.hail.is/u/wmcfadden)\ **Replies:** 3\ **Last updated:** [June 9, 2017, 9:58pm UTC](https://discuss.hail.is/t/importing-vcfs-where-gt-is-0-2-but-ad-has-only-2-entries/234 "2017-06-09T21:58:04Z")
I'm getting an error in after hc.import\_vcf due to the check on whether the number of fields in AD matches the number of alleles implied from GT. I'm just using vcf-merge to combine raw GATK calls so I assume that… --- ## [FileNotFoundException when reading VDS](https://discuss.hail.is/t/filenotfoundexception-when-reading-vds/228)
**Author:** [@Joel\_Thibault](https://discuss.hail.is/u/Joel_Thibault)\ **Replies:** 4\ **Last updated:** [June 7, 2017, 4:23pm UTC](https://discuss.hail.is/t/filenotfoundexception-when-reading-vds/228 "2017-06-07T16:23:20Z")
Continuing my adventures from earlier I am now successfully submitting Spark jobs from my Docker container to DataProc. Now I'm attempting to run Hail commands in Pyspark. Not sure if the problem I seeing is a result o… --- ## [Running Hail with a remote Spark](https://discuss.hail.is/t/running-hail-with-a-remote-spark/227)
**Author:** [@Joel\_Thibault](https://discuss.hail.is/u/Joel_Thibault)\ **Replies:** 10\ **Last updated:** [June 2, 2017, 2:10pm UTC](https://discuss.hail.is/t/running-hail-with-a-remote-spark/227 "2017-06-02T14:10:07Z")
Hello. Can I tell Hail to use a remote Spark installation? All of the examples I've seen require specifying local paths in the configuration, but I would like my Hail installation on a different host. I'm new to Spark… --- ## [EnvironmentError: no Hail context initialized, create one first](https://discuss.hail.is/t/environmenterror-no-hail-context-initialized-create-one-first/223)
**Author:** [@kbchoi-jax](https://discuss.hail.is/u/kbchoi-jax)\ **Replies:** 7\ **Last updated:** [May 18, 2017, 10:43pm UTC](https://discuss.hail.is/t/environmenterror-no-hail-context-initialized-create-one-first/223 "2017-05-18T22:43:10Z")
Trying to run Hail locally, and followed the instruction in "Get Started" documentation at https://hail.is/hail/getting\_started.html. Unfortunately I got the following EnvironmentError. Any comment would be greatly appre… --- ## [New hail and gnomAD, setup help is needed badly :-)](https://discuss.hail.is/t/new-hail-and-gnomad-setup-help-is-needed-badly/208)
**Author:** [@eilalan](https://discuss.hail.is/u/eilalan)\ **Replies:** 19\ **Last updated:** [May 17, 2017, 6:35pm UTC](https://discuss.hail.is/t/new-hail-and-gnomad-setup-help-is-needed-badly/208 "2017-05-17T18:35:59Z")
Hello, I have downloaded the gnomAD files to my cloud bucket and go the following tree: https://console.cloud.google.com/storage/browser/gnomad-public/release-170228 I would like to work with data at the format of VD… --- ## [Import\_table Error in python](https://discuss.hail.is/t/import-table-error-in-python/218)
**Author:** [@lv-xy11](https://discuss.hail.is/u/lv-xy11)\ **Replies:** 2\ **Last updated:** [May 17, 2017, 4:58am UTC](https://discuss.hail.is/t/import-table-error-in-python/218 "2017-05-17T04:58:03Z")
Hi: I come from DCH in China. I try to import table using Hail hc.import\_table command like this: from hail import \* hc=HailContext() table=hc.import\_table('1000Genomes.sample\_annotations',impute=True).key\_by('Sample… --- ## [Another try to extract samples - new approach](https://discuss.hail.is/t/another-try-to-extract-samples-new-approach/216)
**Author:** [@eilalan](https://discuss.hail.is/u/eilalan)\ **Replies:** 5\ **Last updated:** [May 16, 2017, 11:03pm UTC](https://discuss.hail.is/t/another-try-to-extract-samples-new-approach/216 "2017-05-16T23:03:31Z")
Hi, I took a day off from Hail :=) and restarted today my efforts to extract samples by the below code. The idea is simple, get the list of sample IDs, read each sample annotation and write it to a file. However, samp… --- ## [Vds.export\_vcf fires error](https://discuss.hail.is/t/vds-export-vcf-fires-error/215)
**Author:** [@eilalan](https://discuss.hail.is/u/eilalan)\ **Replies:** 1\ **Last updated:** [May 16, 2017, 9:50am UTC](https://discuss.hail.is/t/vds-export-vcf-fires-error/215 "2017-05-16T09:50:26Z")
hello, Any idea why the following code fires the below error? hc = HailContext() vds = hc.read(‘gs://data\_gnomad\_orielresearch/gnomad.exomes.r2.0.1.sites.Y.vds’) vds.export\_vcf(“gs://data\_gnomad\_orielresearch/vdsToCS… --- ## [Simple rare variant burden testing with Fisher exact test](https://discuss.hail.is/t/simple-rare-variant-burden-testing-with-fisher-exact-test/210)
**Author:** [@Mitja](https://discuss.hail.is/u/Mitja)\ **Replies:** 0\ **Last updated:** [May 12, 2017, 7:10pm UTC](https://discuss.hail.is/t/simple-rare-variant-burden-testing-with-fisher-exact-test/210 "2017-05-12T19:10:19Z")
Hail has a general way of performing burden tests within regression framework. http://discuss.hail.is/t/logistic-regression-burden-tests/206. However in rare variant analysis the counts are often so low that some regr… --- ## [HailContext(sc) error](https://discuss.hail.is/t/hailcontext-sc-error/212)
**Author:** [@eilalan](https://discuss.hail.is/u/eilalan)\ **Replies:** 8\ **Last updated:** [May 14, 2017, 6:47pm UTC](https://discuss.hail.is/t/hailcontext-sc-error/212 "2017-05-14T18:47:04Z")
\[6\]: hc = hail.HailContext() Py4JJavaError Traceback (most recent call last) in () ----\> 1 hc = hail.HailContext() /home/eila/hail/python/hail/context.pyc in init(self, sc, appName, master… --- ## [Reading VDS from google bucket fires error](https://discuss.hail.is/t/reading-vds-from-google-bucket-fires-error/209)
**Author:** [@eilalan](https://discuss.hail.is/u/eilalan)\ **Replies:** 9\ **Last updated:** [May 13, 2017, 1:04am UTC](https://discuss.hail.is/t/reading-vds-from-google-bucket-fires-error/209 "2017-05-13T01:04:57Z")
Hello, I am running the following commend: In \[4\]: hc.read(‘gs://data/gnomad.exomes.r2.0.1.sites.autosomes.vds’).count() and getting FatalError: IOException: No FileSystem for scheme: gs any idea why there is an issu… --- ## [Import\_vcf() on databricks results in NoClassDefFoundError](https://discuss.hail.is/t/import-vcf-on-databricks-results-in-noclassdeffounderror/202)
**Author:** [@Shiva](https://discuss.hail.is/u/Shiva)\ **Replies:** 2\ **Last updated:** [May 8, 2017, 10:50pm UTC](https://discuss.hail.is/t/import-vcf-on-databricks-results-in-noclassdeffounderror/202 "2017-05-08T22:50:08Z")
Hi, I am just getting started with hail. I set up the libraries and cluster configuration as mentioned in the tutorial. When I run: vds = hc.import\_vcf(vcf\_path), I get the following error: java.lang.NoClassDefFo… --- ## [Import\_vcf() for tutorial data fails](https://discuss.hail.is/t/import-vcf-for-tutorial-data-fails/199)
**Author:** [@Jens\_Reeder](https://discuss.hail.is/u/Jens_Reeder)\ **Replies:** 2\ **Last updated:** [May 3, 2017, 10:53pm UTC](https://discuss.hail.is/t/import-vcf-for-tutorial-data-fails/199 "2017-05-03T22:53:59Z")
Hi I am trying to get started with hail and run the tutorial. I am running Spark 2.0.1 and hail devel-a51a4c3 on a HPC via a jupyter notebook. After some tinkering with the configs I can succesfully create a hail con… --- ## [Fresh install fails eigSymDSuite and LinearMixedRegressionSuite tests](https://discuss.hail.is/t/fresh-install-fails-eigsymdsuite-and-linearmixedregressionsuite-tests/169)
**Author:** [@CanWood](https://discuss.hail.is/u/CanWood)\ **Replies:** 6\ **Last updated:** [May 3, 2017, 10:25pm UTC](https://discuss.hail.is/t/fresh-install-fails-eigsymdsuite-and-linearmixedregressionsuite-tests/169 "2017-05-03T22:25:19Z")
Hi folks, I've just done a fresh install of spark-2.0.2 and hail from today's git. On executing the Hail tests, all but two passed: LinearMixedRegressionSuite.lmmLargeExampleTest and eigSymDSuite.eigSymTest. On rev… --- ## [Using Hail with Jupyter Notebooks on Google Cloud](https://discuss.hail.is/t/using-hail-with-jupyter-notebooks-on-google-cloud/196)
**Author:** [@Liam\_Abbott](https://discuss.hail.is/u/Liam_Abbott)\ **Replies:** 0\ **Last updated:** [April 29, 2017, 7:44pm UTC](https://discuss.hail.is/t/using-hail-with-jupyter-notebooks-on-google-cloud/196 "2017-04-29T19:44:44Z")
Out of date warning (May 11, 2018): This post is extremely dated, and doesn’t really apply to use of Hail in 2018. The cloudtools repository README is more accurate, and is updated somewhat regularly. Hail with Jupyter … [Previous page](https://discuss.hail.is/c/help-01/6.md?page=4) [Next page](https://discuss.hail.is/c/help-01/6.md?page=6)