Determining how to store secrets for kubernetes has always been a contentious issue. With many different ideas about best practices and how people should go about storing them. Adding to the confusion is the fact that one of the biggest and most important apps associated with the modern kubernetes workflow, argoCD, has taken an unopinionated stance on the subject. Saying that there is no one-size-fits-all solution and that each use case is different.

In my opinion there are 2 ways of handling them. The first is storing and managing them in git, this is generally accomplished via something like SOPS. The second way is storing them in a secrets manager and then referencing them when needed. Of course, there are pros and cons to both but today I will be focusing on the first way, how to manage secrets when you want to store them in git.

My preferred way of doing this is using sealed-secrets, which is a kubernetes controller and tool all-in-one. It works by encrypting your secret into a “SealedSecret”, which is safe to store and can then only be decrypted by the controller running in the kubernetes cluster. Let me show you how to get started!

Installation⌗

To install the Controller we are going to use the officially supported helm chart:

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets

We will be making some updates to the default values of the helm chart. First, rename the controller to sealed-secrets-controller because this is what kubeseal automatically expects. Secondly, install it into the namespace where your app will be, here we will be using dev:

helm install sealed-secrets -n dev --set-string fullnameOverride=sealed-secrets-controller sealed-secrets/sealed-secrets

Verify that the pod is up and running:

kubectl get pods -n dev

To install Kubeseal on linux we can use:

wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.19.4/kubeseal-0.19.4-linux-amd64.tar.gz
tar -xvzf kubeseal-0.19.4-linux-amd64.tar.gz kubeseal
sudo install -m 755 kubeseal /usr/local/bin/kubeseal

To install it on another OS refer to the sealed-secrets docs. We are now ready to start creating our own secrets!

Usage⌗

Create a kubernetes secret with the namespace included. This will only be written to a local file named kubernetes-secret.yaml as we are using the flag --dry-run=client

 kubectl create secret generic my-secret --namespace dev --from-env-file my-secret-file.env --dry-run=client -o yaml > my-secret.yaml

Seal it using kubeseal and make sure to explicitly pass in the namespace where the controller was installed (dev) since the name and the namespace of the secret and SealedSecret must match.

kubeseal --controller-namespace dev  < my-secret.yaml > my-secret-sealed.yaml

Now all we have to do is apply the secret to the cluster:

kubectl create -f my-sealed-secret.yaml

And there we have it! The secret should be deployed to the cluster and it is now safe to store in a public repository.

This is a great solution if you want to be able to store your kubernetes secrets alongside your other kubernetes configuration files. Or you do not want to store them in an external secrets manager.