Incompatible file format ,supported version: 1.1.0, found 1.2.0

Hail Query & hailctl
1

The following error occurs when trying to read table from bucket,
table1 = hl.read_table(‘gs://…ht’),

I created the cluster using hailctl as hailctl dataproc --beta start hailjupy --vep GRCh37 --optional-components=ANACONDA,JUPYTER --enable-component-gateway --bucket bucketname --project projectname --region us-central1

FatalError: HailException: incompatible file format when reading: gs://gnomad-public/release/3.0/ht/genomes/gnomad.genomes.r3.0.sites.ht
supported version: 1.1.0, found 1.2.0

Java stack trace:
is.hail.utils.HailException: incompatible file format when reading: gs://…ht
supported version: 1.1.0, found 1.2.0
at is.hail.utils.ErrorHandling$class.fatal(ErrorHandling.scala:9)
at is.hail.utils.package$.fatal(package.scala:74)
at is.hail.variant.RelationalSpec$.readMetadata(MatrixTable.scala:54)
at is.hail.variant.RelationalSpec$.readReferences(MatrixTable.scala:71)
at is.hail.variant.ReferenceGenome$.fromHailDataset(ReferenceGenome.scala:586)
at is.hail.variant.ReferenceGenome.fromHailDataset(ReferenceGenome.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)

Hail version: 0.2.23-aaf52cafe5ef
Error summary: HailException: incompatible file format when reading: gs://…ht
supported version: 1.1.0, found 1.2.0

2

This means you need to update your Hail installation. Hail can read files back-compatibly (a new version can read files made in an old version of 0.2) but not forward-compatibly (an older version cannot necessarily read files written in a newer version of Hail).

I’ll also fix the error message to indicate that you need to upgrade.

3

I upgraded my hail but another error occur and now i am unable to create hail cluster on cloud… i tried creating normal cluster using hailctl dataproc start cluster1, which was successfully created but when i tried to run hailctl dataproc --beta start hailjupy --vep GRCh37 --optional-components=ANACONDA,JUPYTER --enable-component-gateway --bucket bucketname --project projectname --region us-central1 ,then it gives error sometimes timeout error and sometimes other…

kindly help me with this issue…

/us-central1/operations/b5092f91-adb0-3ed3-83cf-de5f14335f01] failed: Multiple Errors:

  • Timeout waiting for instance hailjupy-m to report in.
  • Timeout waiting for instance hailjupy-w-0 to report in.
  • Timeout waiting for instance hailjupy-w-1 to report in…
    Traceback (most recent call last):
    File “/home/…/env/bin/hailctl”, line 8, in
    sys.exit(main())
    File “/home/…/env/lib/python3.7/site-packages/hailtop/hailctl/main.py”, line 94, in main
    Waiting for cluster creation operation…done.

M,hdfs:dfs.replication=1,dataproc:dataproc.logging.stackdriver.enable=false,dataproc:dataproc.monitoring.stackdriver.enable=false,spark:spark.driver.memory=41g’, ‘–initialization-actions=gs://hail-common/hailctl/dataproc/0.2.27/init_notebook.py,gs://hail-common/hailctl/dataproc/0.2.27/vep-GRCh37.sh’, ‘–metadata=^|||^WHEEL=gs://hail-common/hailctl/dataproc/0.2.27/hail-0.2.27-py3-none-any.whl|||PKGS=aiohttp>=3.6,<3.7|aiohttp_session>=2.7,<2.8|asyncinit>=0.2.4,<0.3|bokeh>1.1,<1.3|decorator<5|gcsfs==0.2.1|hurry.filesize==0.9|nest_asyncio|numpy<2|pandas>0.24,<0.26|parsimonious<0.9|PyJWT|python-json-logger==0.1.11|requests>=2.21.0,<2.21.1|scipy>1.2,<1.4|tabulate==0.8.3’, ‘–master-machine-type=n1-highmem-8’, ‘–master-boot-disk-size=100GB’, ‘–num-master-local-ssds=0’, ‘–num-preemptible-workers=0’, ‘–num-worker-local-ssds=0’, ‘–num-workers=2’, ‘–preemptible-worker-boot-disk-size=200GB’, ‘–worker-boot-disk-size=200GB’, ‘–worker-machine-type=n1-highmem-8’, ‘–zone=us-central1-b’, ‘–initialization-action-timeout=20m’, ‘–project=…’, ‘–bucket=…’, ‘–labels=creator=…_gmail_com’, ‘–optional-components=ANACONDA,JUPYTER’, ‘–enable-component-gateway’, ‘–region’, ‘us-central1’]’ returned non-zero exit status 1.