A free k8s cluster on Oracle Cloud via Terraform and Ansible - Part 2.5, Working, but ARM makes it weird

Updated: at 08:44 PM (3 min read)

Building on the theme of recent posts, I was trying to deploy a dummy Spring Boot application and getting an error about the log format being an issue. I forgot we are running on ARM. Time to create a GitLab Runner.

Table of contents

Open Table of contents

Deployment does not start, it is all in the architecture

So today I learned that Docker images are architecture dependent, they have binaries added from the host system. I was using the shared Runners on GitLab, and looking thought the list, they are all AMD64. This presented an opportunity, could I start my own runner on my shiny new K8s cluster, to build my docker image, to run that on my shiny new k8s cluster?

Take the Helm

In the official docs, GitLab say to use their helm chart to install the runner and get it configured. So to the command line!

  1. Create a namespace to keep the runner in with kubectl create ns gitlab-runner, or use the GUI on Lens.
  2. Add the Helm repo from GitLab with helm repo add gitlab https://charts.gitlab.io.
kubectl create ns gitlab-runner

helm repo add gitlab https://charts.gitlab.io
  1. Reconnect (I raised a bug on this) Lens to the cluster and head to the charts tab.
  2. Find the GitLab Runner chart, click it and click install to get the code window for the config.
  3. Add the following items to the config
    1. gitlabUrl this for me was https://gitlab.com
    2. rbac.create set to true - I think I had this set to a map, but this took a lot of revisions (13) to find the right config and I seem to have lost the link to them.
    3. runnerToken in the GitLab UI, in your project/group, go to the Settings -> CI/CD -> Runners section and click add runner, this will give you a token you need to add here.
    4. The runners.config value was the hardest to get right. I used the below from various parts of the GitLab docs to get it working.
   runners:
  cache: {}
  config: |
    [[runners]]
    [runners.kubernetes]
    namespace = "{{.Release.Namespace}}"
    image = "ubuntu:20.04"
    privileged = true
    [[runners.kubernetes.volumes.config_map]]
    name = "docker-client-config"
    mount_path = "/root/.docker/config.json"
    sub_path = "config.json"
    [[runners.kubernetes.volumes.empty_dir]]
    name = "docker-certs"
    mount_path = "/certs/client"
    medium = "Memory"
    [runners.kubernetes.node_selector]
    "kubernetes.io/arch" = "arm64"
  1. securityContext This might not have been needed, I think you only need to add the privileged to the config, but I have not gone back to test it
securityContext:
  allowPrivilegeEscalation: true
  privileged: true

  1. Click install and wait, but there is more.

  2. To allow the runner to pull from docker.io to get its container images, you need to add an authentication token.

    1. base64 encode your docker hub username and password printf "USERNAME:PASSWORD" | openssl base64 -A - I tried an auth token but that didn’t work.
    2. add this to a config.yaml file for authentication to docker
    { auths: { https://index.docker.io/v1/: { auth: BASE_64_STRING } } }
    1. add the config map.
printf "USERNAME:PASSWORD" | openssl base64 -A

kubectl create configmap docker-client-config --namespace gitlab_runner --from-file config.json
  1. it should look something like this:
apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-client-config
  namespace: gitlab_runner
data:
  config.json: "{
    "auths\": {
    t\"https://index.docker.io/v1/\": {\n\t\t\t\"auth\": \"BASE_64_STRING\"\n\t\t}\n\t}\n}"
  1. In the runner GUI I tagged this one as arm64, then we can tag jobs in the ci file as the same and they will run on our cluster!

Resources

https://github.com/nce/oci-free-cloud-k8s/blob/main/terraform/infra https://platform9.com/learn/v1.0/tutorials/traefik-ingress#step-2---expose-traefik-dashboard