How to creates Kubernetes jobs with Python

1 min read

Jan 28, 2019 12:00:00 AM

In this blog post I will do a quick guide, with some code examples, on how to deploy a Kubernetes Job programmatically, using Python as the language of choice. For this I’m using GKE (Google Kubernetes Engine), logging via StackTrace and haveana image available on Google Container Registry. The architecture should be something like this:

The code that I created:

A Dockerfile for my container
A Python App that has the code to run (this will be the Job)

How does all this works?

Commit the code to the GCP Source Code Repositories
A CloudBuild trigger (docs: https://cloud.google.com/cloud-build/docs/) that creates the container
Create a trigger (can be a CronJob) that runs the code that deploys the Job.
1. For this exercise, I’m going to trigger the Job creation from my own laptop.

Now the code. The difficult part here was dealing a bit with the documentation. For this code I used the following library: https://github.com/kubernetes-client/python/tree/master/kubernetes It provides the Kubernetes abstraction layer and greatly simplifies the work. Now, to deploy a Kubernetes Job, our code needs to build the following objects:

Job object
- Contains a metadata object
- Contains a job spec object
  - Contains a pod template object
    - Contains a pod template spec object
      - Contains a container object

You can walk through the Kubernetes library code and check how it gets and forms the objects. Also, don’t forget that all this is based on the official Job specification ( https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/#writing-a-job-spec) Without much else to say, you can check the full code here:

On this page

Ready to unlock value from your data?

With Pythian, you can accomplish your data transformation goals and more.

Speak with Pythian consultants now →

What We Do

How to creates Kubernetes jobs with Python

Share this

Share this

More resources

Configuring OID 11.1.1.6 Connector in OIM 11.1.2

How to build a cost-effective serverless blogging platform on GCP - Part 1

Oracle upgrade failures due to METHOD_OPT and XDBCONFIG

Ready to unlock value from your data?