Google Cloud Platform quick tip: Cloud Shell boost mode for Docker development

Written by Pythian Marketing | Apr 23, 2018 4:00:00 AM

I explained in an earlier post how I am using GCP's Cloud Shell for all of my command-line needs all the time. Whenever you start your cloud shell, Google Cloud Platform (GCP) spins up a small compute instance with the Cloud Shell image for you. This is using a g1-small sized compute engine which provides 0.5 vCPUs and 1.7GB of memory. That's more than enough for most command-line tasks. But you'll hit these limits when you also want to run programs or even containers locally on the Cloud Shell instance.

Engaging the turbo

If you temporarily need more compute power, you can enable Cloud Shell's "Boost Mode" through the menu on the top right. It just takes one click to enable this. This will restart your Cloud Shell on a new compute instance of n1-standard-1 size. This more than doubles your compute resources to 1 vCPU and 3.75GB of memory which should be enough for a lot of use cases. This boost will last 24 hours. When you re-connect to Cloud Shell after that, it will provision the smaller instance type for you until you request the boost mode again.

Docker in Cloud Shell

One of the things I do in Cloud Shell is developing and debugging docker containers. For one of my presentations about stream processing, I am using a container to run a playground with mysql, debezium, kafka, ksql and grafana in one handy image. Fitting all of these into the 1.7GB RAM, the default Cloud Shell provides, is cutting it very close and not a great experience. But with the help of Boost Mode it just flies. Here is the memory info before and after I enabled Boost Mode:

rost@cloudshell:~$ free -h
  total used free shared buff/cache available
 Mem: 1.7G 195M 1.2G 80K 240M 1.3G
 Swap: 767M 143M 624M
 
 rost@cloudshell:~$ free -h
  total used free shared buff/cache available
 Mem: 3.6G 225M 2.9G 1.3M 474M 3.2G
 Swap: 767M 0B 767M

After enabling Boost Mode, just run the docker container as shown below. Docker images are stored in /var/lib/docker which is not part of the 5GB persistent home directory. I see this as a good thing as otherwise your home directory would fill up quickly. So chances are that the docker image has to get pulled again when you have not used Cloud Shell for a while and a new compute instance was provisioned for you. I am forwarding port 3000 (grafana) from the container to 8080 of the cloud shell instance. This allows me to use the convenient "web preview" feature in Cloud Shell without having to worry about copying ephemeral IP addresses and such.

rost@cloudshell:~$ docker run -p 8080:3000 -ti brost/stream-etl:ksql bash
 Unable to find image 'brost/stream-etl:ksql' locally
 ksql: Pulling from brost/stream-etl
 ad74af05f5a2: Pulling fs layer
 d02e292e7b5e: Pull complete
 8de7f5c81ab0: Pull complete
 ed0b76dc2730: Pull complete
 cfc44fa8a002: Pull complete
 [...]
 Digest: sha256:eeb4453503327912744bb03f0b1d769a3d88dc0ec297dede3fb945e3fb0545fd
 Status: Downloaded newer image for brost/stream-etl:ksql
 [ ok ] Starting MySQL database server: mysqld ..
 [info] Checking for tables which need an upgrade, are corrupt or were
 not closed cleanly..
 [ ok ] Starting Grafana Server:.
 [ ok ] Starting Elasticsearch Server:.
 Starting zookeeper
 zookeeper is [UP]
 Starting kafka
 kafka is [UP]
 Starting schema-registry
 schema-registry is [UP]
 Starting kafka-rest
 kafka-rest is [UP]
 Starting connect
 connect is [UP]
 root@ac86f0078c9b:/#

Conclusion

Google's Cloud Shell is already a nifty tool for doing command line work from any device in the cloud. When you are pushing the default resource limits of the instance, you can double CPU and memory capacity temporarily by restarting your Cloud Shell in Boost mode.

Google Cloud Consulting Services

Ready to optimize your use of Google Cloud's AI tools?

View full post