Part 2 — Running a job
Kubernetes allows you to run almost any type of application. In this guide we’ll start with running a one-off job. Jobs in Kubernetes are similar to jobs in Nerd: they will run an application that’s packaged as a Docker Image. Kubernetes provides more flexibility to configure how jobs are run, but unlike Nerd your input and output data isn’t automatically handled.
We’ve created an animal classifier (using machine learning) and packaged it as an Image. It will get a picture from the internet and detect whether it’s a dog, cat, monkey or a squirrel. We’ll use it to show you how to use Kubernetes.
Creating a job
To run a job in you’ll need to tell Kubernetes to create a Job. When doing so, you have to provide a Spec file with specifications such as a name, the application Image name and application-specific configuration.
We’ve written a Job Spec file for you. You can use it to analyze our example image by running:
$ kubectl create -f https://www.nerdalize.com/docs/kubernetes/getting-started/tensorflow-example.yaml
Viewing progress & log output
You’ve started a job now. To see your jobs and their status, run:
$ kubectl get jobs
This shows a list of all of your jobs and, amongst other things, whether they’ve succeeded. kubectl shows a number because you can configure your job to run multiple times.
Each time a job runs (including when it fails) Kubernetes will create a Pod to run your application in. You can therefore view the logs of your application by looking at the Pod that ran it.
You’ll first need to list the Pods that your job created. To do so you can get a list of pods filtered using the job name by running:
$ kubectl get pods --show-all --selector=job-name=tensorflow-example
--show-all argument is important because kubectl won’t show completed pods by default.
You can then get logs for a Pod, by using its name in the following command and running it:
$ kubectl logs <name-of-pod>
Customizing your job
Many applications use environment variables as a way to configure options or provide some input data. To give Kubernetes the environment variables, you’ll need to update the Job Spec.
Our animal classifier, called nerdalize/tensorflow-example, allows you to analyze a different image by providing a URL. To do so make a customized Job Spec by creating a file on your computer with the following contents and updating the
value for the
apiVersion: batch/v1 kind: Job metadata: name: tensorflow-example-custom spec: template: spec: containers: - name: tf image: nerdalize/tensorflow-example env: - name: SUBJECT value: "<your-image-url>" resources: requests: memory: "1Gi" cpu: "250m" limits: memory: "1Gi" cpu: "500m" restartPolicy: Never backoffLimit: 4
To start the animal classifier with your configuration, run:
$ kubectl create -f <path-to-spec-file>
The Job Spec provides for many more options to customize how your job runs. The official Kubernetes documentation provides an overview.
That’s how to run a job
Awesome! You’ve created jobs and viewed job logs by using
kubectl get jobs,
kubectl get pods &
kubectl logs. If you’ve tried Nerd before, you might have noticed Nerd is very easy to use. Kubernetes is a bit more advanced, but super flexible & powerful. So, let’s continue to the final part to find out where to learn more about what you can do using Kubernetes and how to do it.