# Hailctl breaks with latest GCP SDK 294.0.0

**URL:** <https://discuss.hail.is/t/hailctl-breaks-with-latest-gcp-sdk-294-0-0/1328>\
**Category:** Hail Query & hailctl\
**Created:** [March 11, 2020, 8:18pm UTC](https://discuss.hail.is/t/hailctl-breaks-with-latest-gcp-sdk-294-0-0/1328 "2020-03-11T20:18:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mwilson](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/mwilson/32/376_2.png) [@mwilson](https://discuss.hail.is/u/mwilson)\
**Post date:** [March 11, 2020, 8:18pm UTC](https://discuss.hail.is/t/hailctl-breaks-with-latest-gcp-sdk-294-0-0/1328/1 "2020-03-11T20:18:03Z")

</div>

Hello,

I recently updated my GCP SDK to 294.0.0 and received the following error when starting a cluster with hailctl.

> wma13-a4c:~ 250 1 $ hailctl dataproc start mw --properties=spark:spark.speculation=true --num-preemptible-workers=2 --worker-machine-type=n1-highmem-8 --max-idle=120m --packages slackclient==2.0.0,websocket-client,sklearn,statsmodels,scikit-learn,hdbscan,matplotlib,google-cloud-bigquery,gnomad  
> Your active configuration is: [seqr-work]  
> gcloud beta dataproc clusters create   
> mw   
> –image-version=1.4-debian9   
> –properties=spark:spark.task.maxFailures=20,spark:spark.driver.extraJavaOptions=-Xss4M,spark:spark.executor.extraJavaOptions=-Xss4M,hdfs:dfs.replication=1,dataproc:dataproc.logging.stackdriver.enable=false,dataproc:dataproc.monitoring.stackdriver.enable=false,spark:spark.speculation=true,spark:spark.driver.memory=41g   
> –initialization-actions=gs://hail-common/hailctl/dataproc/0.2.33/init\_notebook.py   
> –metadata=^|||^WHEEL=gs://hail-common/hailctl/dataproc/0.2.33/hail-0.2.33-py3-none-any.whl|||PKGS=aiohttp\>=3.6,\<3.7|aiohttp\_session\>=2.7,\<2.8|asyncinit\>=0.2.4,\<0.3|bokeh\>1.1,\<1.3|decorator\<5|gcsfs==0.2.1|humanize==1.0.0|hurry.filesize==0.9|nest\_asyncio|numpy\<2|pandas\>0.24,\<0.26|parsimonious\<0.9|PyJWT|python-json-logger==0.1.11|requests\>=2.21.0,\<2.21.1|scipy\>1.2,\<1.4|tabulate==0.8.3|tqdm==4.42.1|slackclient==2.0.0|websocket-client|sklearn|statsmodels|scikit-learn|hdbscan|matplotlib|google-cloud-bigquery|gnomad   
> –master-machine-type=n1-highmem-8   
> –master-boot-disk-size=100GB   
> –num-master-local-ssds=0   
> –num-preemptible-workers=2   
> –num-worker-local-ssds=0   
> –num-workers=2   
> –preemptible-worker-boot-disk-size=40GB   
> –worker-boot-disk-size=40GB   
> –worker-machine-type=n1-highmem-8   
> –zone=us-central1-b   
> –initialization-action-timeout=20m   
> –labels=creator=mwilson\_broadinstitute\_org   
> –max-idle=120m  
> Starting cluster ‘mw’…  
> WARNING: The `--num-preemptible-workers` flag is deprecated. Use the `--num-secondary-workers` flag instead.  
> WARNING: The `--preemptible-worker-boot-disk-size` flag is deprecated. Use the `--secondary-worker-boot-disk-size` flag instead.  
> ERROR: (gcloud.beta.dataproc.clusters.create) Error parsing [cluster].  
> The [cluster] resource is not properly specified.  
> Failed to find attribute [region]. The attribute can be set in the following ways:
> 
> - provide the argument [–region] on the command line
> - set the property [dataproc/region]  
> Traceback (most recent call last):  
> File “/Users/mwilson/anaconda3/bin/hailctl”, line 8, in   
> sys.exit(main())  
> File “/Users/mwilson/anaconda3/lib/python3.6/site-packages/hailtop/hailctl/ **main**.py”, line 94, in main  
> cli.main(args)  
> File “/Users/mwilson/anaconda3/lib/python3.6/site-packages/hailtop/hailctl/dataproc/cli.py”, line 107, in main  
> jmp[args.module].main(args, pass\_through\_args)  
> File “/Users/mwilson/anaconda3/lib/python3.6/site-packages/hailtop/hailctl/dataproc/start.py”, line 202, in main  
> sp.check\_call(cmd)  
> File “/Users/mwilson/anaconda3/lib/python3.6/subprocess.py”, line 311, in check\_call  
> raise CalledProcessError(retcode, cmd)  
> subprocess.CalledProcessError: Command ‘[‘gcloud’, ‘beta’, ‘dataproc’, ‘clusters’, ‘create’, ‘mw’, ‘–image-version=1.4-debian9’, ‘–properties=spark:spark.task.maxFailures=20,spark:spark.driver.extraJavaOptions=-Xss4M,spark:spark.executor.extraJavaOptions=-Xss4M,hdfs:dfs.replication=1,dataproc:dataproc.logging.stackdriver.enable=false,dataproc:dataproc.monitoring.stackdriver.enable=false,spark:spark.speculation=true,spark:spark.driver.memory=41g’, ‘–initialization-actions=gs://hail-common/hailctl/dataproc/0.2.33/init\_notebook.py’, ‘–metadata=^|||^WHEEL=gs://hail-common/hailctl/dataproc/0.2.33/hail-0.2.33-py3-none-any.whl|||PKGS=aiohttp\>=3.6,\<3.7|aiohttp\_session\>=2.7,\<2.8|asyncinit\>=0.2.4,\<0.3|bokeh\>1.1,\<1.3|decorator\<5|gcsfs==0.2.1|humanize==1.0.0|hurry.filesize==0.9|nest\_asyncio|numpy\<2|pandas\>0.24,\<0.26|parsimonious\<0.9|PyJWT|python-json-logger==0.1.11|requests\>=2.21.0,\<2.21.1|scipy\>1.2,\<1.4|tabulate==0.8.3|tqdm==4.42.1|slackclient==2.0.0|websocket-client|sklearn|statsmodels|scikit-learn|hdbscan|matplotlib|google-cloud-bigquery|gnomad’, ‘–master-machine-type=n1-highmem-8’, ‘–master-boot-disk-size=100GB’, ‘–num-master-local-ssds=0’, ‘–num-preemptible-workers=2’, ‘–num-worker-local-ssds=0’, ‘–num-workers=2’, ‘–preemptible-worker-boot-disk-size=40GB’, ‘–worker-boot-disk-size=40GB’, ‘–worker-machine-type=n1-highmem-8’, ‘–zone=us-central1-b’, ‘–initialization-action-timeout=20m’, ‘–labels=creator=mwilson\_broadinstitute\_org’, ‘–max-idle=120m’]’ returned non-zero exit status 1.

It looks like google updated the flags for preemptibles. I reverted back to 259.0.0 but figured I’d let you guys know. Considering I haven’t updated the google SDK in awhile I’m not sure when this change happened, I couldn’t find any documentation on it in a brief search.

---

<div class="post-metadata">

**Author:** ![mwilson](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/mwilson/32/376_2.png) [@mwilson](https://discuss.hail.is/u/mwilson)\
**Post date:** [March 11, 2020, 8:19pm UTC](https://discuss.hail.is/t/hailctl-breaks-with-latest-gcp-sdk-294-0-0/1328/2 "2020-03-11T20:19:10Z")

</div>

Sorry just realized the main cause of the error was the region flag which I think was addressed elsewhere…

---

<div class="post-metadata">

**Author:** ![tpoterba](https://yyz2.discourse-cdn.com/flex036/user_avatar/discuss.hail.is/tpoterba/32/61_2.png) [@tpoterba](https://discuss.hail.is/u/tpoterba)\
**Post date:** [March 11, 2020, 8:19pm UTC](https://discuss.hail.is/t/hailctl-breaks-with-latest-gcp-sdk-294-0-0/1328/3 "2020-03-11T20:19:39Z")

</div>

The worker stuff is just warnings, yeah. Next release we’re also going to make sure you have region set either as an arg or in config.