Using custom kernel images

Self-hosted deployments of Hex can build their own kernel images to install custom packages, configure environment variables, add files and more. Using a custom kernel requires some extra setup which will be covered below.

Building a custom kernel

All custom kernels must be based off the default kernel that matches the version of Hex that is deployed. The base kernel images and their tags can be found at the Hex Public Docker Hub. If you do not have an existing process for building docker images, there are a variety of tools and services you can use to set one up. Alternatively, you can also build docker images from your local machine.

Commonly used docker build services:

Once you have chosen a tool for building your custom kernel image, you can now set up your Dockerfile to make any changes or additions. The only requirements are that you do not change the version of ipykernel and that the ENTRYPOINT calls /ipython/launch_kernel.sh /ipython-shared/kernel.json /ipython/config.py. An example Dockerfile is below:

FROM hexinc/python-kernel:0.8.0
COPY ./extra_files /hex/my_files
COPY ./deploy_key ~/.ssh/id_rsa
RUN git clone https://github.com/customer/myrepo.git
RUN pip3 install -e ./myrepo

If you do want to change the ENTRYPOINT make sure that whatever the new entrypoint is calls that as the last line. As an example, here is a bash script that could replace the entrypoint (e.g. ENTRYPOINT [ "myscript.sh" ]):

#!/bin/bash
cd ./myrepo
git pull
curl http://some-api.com/latest-files.json -O /hex/latest-files.json
/ipython/launch_kernel.sh /ipython-shared/kernel.json /ipython/config.py

Publishing the custom kernel

Once you have finished building your custom kernel, you must publish it in a place available to Kubernetes to use. Some common docker image registries are:

Using the custom kernel

To use a custom kernel with Hex, after publishing the custom kernel you must set up the image in the deployment settings in Kots.

Set the docker registry, image, and tag to be what you published to in the previous steps.

Using a private repository

If your Docker Repository is private, you will need to follow a few extra steps currently in order to use the above image.

Settings up Kubernetes Image Pull Secrets

If your Docker Repository is private, you may need to generate a secret token in order to get Kubernetes to be able to access it. Each registry has a different process for generating and creating these secret tokens, but once you have created a token in the same namespace that Hex resides in, add the name into the Hex config.

Some links for generating docker secrets for common docker registries:

Patching with kustomize

Currently, Kots has a bug where private repositories are incorrectly proxied through Replicated, even though the credentials may be correct via the above image pull secrets. In order to fix this, you have to apply a Kustomize patch in the kots UI.

Instructions on how to do this can be found here: https://kots.io/kotsadm/updating/patching-with-kustomize/

Example patch:

Edit the file in /ovelays/downstreams/this-cluster/kustomization.yaml to include an override for your image name which removes the http://proxy.replicated.com/proxy/hex/ portion of the address like so: (Do not include/change the imageTag)

apiVersion: kustomize.config.k8s.io/v1beta1
bases:
- ../../midstream
kind: Kustomization
images:
- name: proxy.replicated.com/proxy/hex/997441504095.dkr.ecr.us-east-2.amazonaws.com/hex/hex-python-kernel
newName: 997441504095.dkr.ecr.us-east-2.amazonaws.com/hex/hex-python-kernel

You should only need to do this once, as long as the image repository + url never changes.