Storing Configuration Data in Kubernetes: Best Practices and Tools

Kubernetes (k8s) is a powerful container orchestration platform that can help you manage your applications at scale. But as you deploy more and more applications on Kubernetes, you’ll need to manage their configuration data too. In this article, we’ll explore the best practices and tools for storing configuration data in Kubernetes.

ConfigMaps: Storing Key-Value Pairs and Configuration Files

One of the simplest ways to store configuration data in Kubernetes is to use ConfigMaps. ConfigMaps are Kubernetes objects that allow you to store key-value pairs or configuration files. You can then mount ConfigMaps as volumes in your containers, inject them as environment variables, or use them as a configuration source for other Kubernetes objects like Deployments and StatefulSets.

To create a ConfigMap, you can use the kubectl create configmap command. Here’s an example:

$ kubectl create configmap my-config --from-literal=foo=bar --from-file=my-confg-file

This creates a ConfigMap called “my-config” with a key-value pair “foo=bar” and a configuration file “my-config-file”. You can then use this ConfigMap in your Kubernetes objects like this:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: my-image
    env:
    - name: FOO
      valueFrom:
        configMapKeyRef:
          name: my-config
          key: foo
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
  volumes:
  - name: config-volume
    configMap:
      name: my-config

This mounts the “my-config” ConfigMap as a volume in the container and sets the environment variable “FOO” to the value of the “foo” key in the ConfigMap.

Secrets: Storing Sensitive Data

If you need to store sensitive data like passwords, API keys, or certificates, you should use Secrets instead of ConfigMaps. Secrets are similar to ConfigMaps but are specifically designed for storing sensitive data. Secrets are encrypted at rest and can be used by applications as environment variables or mounted as volumes in a container.

To create a Secret, you can use the kubectl create secret command. Here’s an example:

$ kubectl create secret generic my-secret --from-literal=username=myuser --from-literal=password=mypassword

This creates a Secret called “my-secret” with two key-value pairs “username=myuser” and “password=mypassword”. You can then use this Secret in your Kubernetes objects like this:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: my-image
    env:
    - name: USERNAME
      valueFrom:
        secretKeyRef:
          name: my-secret
          key: username
    - name: PASSWORD
      valueFrom:
        secretKeyRef:
          name: my-secret
          key: password
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
  volumes:
  - name: secret-volume
    secret:
      secretName: my-secret

This sets the environment variables “USERNAME” and “PASSWORD” to the values of the “username” and “password” keys in the “my-secret” Secret and mounts the Secret as a volume in the container.

The default implementation for Secret in Kubernetes is not safe for storing sensitive data. By default, Secrets are stored in plaintext in etcd, the Kubernetes data store. This means that anyone with access to the etcd cluster can potentially read the Secrets, including sensitive data like passwords, API keys, and certificates.

To mitigate this security risk, it’s important to encrypt the data in Secrets before storing them in etcd. Kubernetes provides two ways to do this: using the --encrypt-data flag or using a custom encryption provider.

Using the --encrypt-data Flag

When you create a Secret in Kubernetes, you can use the --encrypt-data flag to encrypt the data in the Secret before storing it in etcd. This flag encrypts the data using a randomly generated key, which is then encrypted using a key encryption key (KEK) stored in a Kubernetes Secret. This approach provides a basic level of security for Secrets.

Here’s an example of creating an encrypted Secret using the --encrypt-data flag:

$ echo -n "my-password" | base64 
bXktcGFzc3dvcmQ= 

$ kubectl create secret genric my-secret --from-literal=password=bXktcGFzc3dvcmQ= --encrypt-data

This creates a Secret named “my-secret” with a password stored in an encrypted format.

Using a Custom Encryption Provider

If you need more advanced encryption options, you can use a custom encryption provider to encrypt the data in Secrets. Kubernetes allows you to write custom encryption providers using the Kubernetes Extensibility API. This approach allows you to use your own encryption algorithms and keys to encrypt the data in Secrets.

Here’s an example of using a custom encryption provider to encrypt the data in a Secret:

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  annotations:
    kubernetes.io/secret-encryption: "custom"
    my-custom-provider.com/key-id: "my-key-id"
type: Opaque
data:
  password: bXktcGFzc3dvcmQ=

This creates a Secret named “my-secret” with a password stored in an encrypted format using a custom encryption provider. The encryption provider is identified by the “kubernetes.io/secret-encryption” annotation, and the encryption key ID is specified using a custom annotation.

External Configuration Stores: More Complex Configuration Data Management

If you have more complex configuration data management requirements, you may want to consider using external configuration stores like etcd, ZooKeeper, or Consul. These stores can be integrated with Kubernetes using custom controllers or operators.

For example, you can use the etcd-operator to deploy a highly available etcd cluster on Kubernetes and use it to store configuration data for your applications. The etcd-operator provides a Kubernetes API for managing etcd clusters and allows you to define etcd clusters as Kubernetes objects.

Conclusion

Storing configuration data in Kubernetes can be done in several ways, depending on your application’s needs and requirements. You can use ConfigMaps to store key-value pairs and configuration files, Secrets to store sensitive data, Kubernetes API objects to define configuration data as part of object specification, or external configuration stores for more complex configuration data management. By choosing the right approach and tool for your application, you can simplify configuration data management in Kubernetes and make your applications more scalable and reliable.