Scheduling Google Cloud Functions

April 25, 2018

Currently, there is no straightforward way to schedule Google Cloud Functions. It is still possible to achieve this by different means, such as (but not limited to):

deploying Compute Engine instance and setting crontab entry
configuring HTTP/S uptime checks via Stackdriver Monitoring
deploying App Engine cron job

Question mays arise: why would one need to do this at all? There may be different use cases, for example, pulling data from external sources periodically, or triggering internal pipelines based on the schedule, etc. Here's a summary of the options mentioned above:

Compute Engine instance

Deploying separate Compute Engine instance just for this does not sound right. Also, to achieve High Availability, multiple instances have to be deployed, leading to each instance scheduling tasks independently of the others.

Stackdriver Monitoring uptime check

Stackdriver Monitoring uptime checks are performed from all available locations, by default. There is a way to limit checks by one or more location, but some of them, US for example, have multiple regions from where the checks are performed. Again, the function will be iinvoked more often than required.

App Engine cron job

App Engine cron job, on the other side, is one of the recommended ways to implement reliable cron functionality. It comes with its own limitations, such as App Engine cron job can only invoke App Engine endpoints, so an application needs to be deployed as well. By default, it will be deployed in High Availability setup, having 2 instances running.

Implementing Cloud Function scheduling via App Engine cron job

When following instructions assume there is a Google Cloud Platform project named test-gcf-scheduling and service account named deployment with Project Owner permissions created, and service account JSON key has been downloaded under deployment-key.json. Also, sample code repository has been cloned into current working directory: [code language="bash"] git clone https://github.com/ilgarm/test-gcf-scheduling.git cd test-gcf-scheduling [/code] Authorize service account first and enable required services: [code language="bash"] gcloud config set project test-gcf-scheduling gcloud auth activate-service-account --key-file=test-gcf-scheduling-key.json gcloud services enable appengine gcloud services enable cloudfunctions [/code]

Cloud Function

Next, we could create staging bucket to upload functions to, otherwise, one will be created by the service. [code language="bash"] gsutil mb -c regional -l us-central1 gs://gcf-test-scheduling-staging cat > gcf-test-scheduling-staging-lifecycle.json << EOL { "rule": [ { "action": {"type": "Delete"}, "condition": {"age": 7} } ] } EOL gsutil lifecycle set gcf-test-scheduling-staging-lifecycle.json gs://gcf-test-scheduling-staging [/code] To deploy the function: [code language="bash"] gcloud beta functions deploy scheduled --source=gcf/scheduled-function --stage-bucket gcf-test-scheduling-staging --memory=128MB --region=us-central1 --trigger-http [/code] A few notes on the sample function. As the function is exposed via HTTP/S it might be important to password protect it, to avoid unexpected costs incurred by unauthorized invocation, especially, if function triggers some heavy processing.

App Engine application

Next, App Engine application needs to be deployed. The current version is deployed to the standard python27 environment with manual scaling of 1 instance, to avoid unnecessary costs. Single B1 instance feets into Free Tier also. The code is straightforward, the only important thing to mention here is the following section: [code language="python"] if self.request.headers.get('X-Appengine-Cron', False) != "true": raise ValueError('Not invoked from the task queue job') [/code] The header is injected into the request if originated from the App Engine cron job. For other requests, the header is stripped out if present. This should be enough to ensure that the request is issued by our cron job. To deploy to App Engine: [code language="bash"] gcloud app deploy gae/scheduler-app/app.yaml gcloud app deploy gae/scheduler-app/cron.yaml [/code] Now, navigate to the Cloud Functions logs, there should be log entries for the function invocation appearing every minute. This example could also be extended to trigger multiple functions on different schedules, implement crontab expression evaluation logic and so on.

Insight and analysis of technology and business strategy