# Interface

## 1. Overview

**K8S Resource Security** is built using the interface provided by **Kubernetes (K8S)**.\
Kubernetes provides this functionality under the title **“Using a KMS provider for data encryption”** in its official documentation.

Relevant information from the official documentation has been excerpted and referenced below.

* Official documentation : <https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/>

### **1-1. KMS Version** <a href="#id-31-kms-version" id="id-31-kms-version"></a>

The **KMS provider** supports the following two versions, as described in the official Kubernetes documentation.

* **KMS v1**: Kubernetes v1.28 \[deprecated]
  * Kubernetes version 1.10.0 or later is required
  * For version 1.29 and later, the v1 implementation of KMS is disabled by default. To enable the feature, set `--feature-gates=KMSv1=true` to configure a KMS v1 provider.
  * Your cluster must use etcd v3 or later
* **KMS v2**: Kubernetes v1.29 \[stable]
  * Your cluster must use etcd v3 or later

### **1-2.** KMS Provider Configuration <a href="#id-31-kms-version" id="id-31-kms-version"></a>

To configure a **KMS provider** in the API server, include a provider of type `kms` in the provider array of the encryption configuration file, and set the following properties accordingly.

* **KMS v1**
  * `apiVersion`: API Version for KMS provider. Leave the value empty or set it to `v1`.
  * `name`: Display name of the KMS plugin. Cannot be changed once set.
  * `endpoint`: Listen address of the gRPC server (KMS plugin). The endpoint is a UNIX domain socket.
  * `cachesize`: Number of data encryption keys (DEKs) to be cached in the clear. When cached, DEKs can be used without another call to the KMS; whereas DEKs that are not cached require a call to the KMS to unwrap.
  * `timeout`: How long should `kube-apiserver` wait for kms-plugin to respond before returning an error (default is 3 seconds).
* **KMS v2**
  * `apiVersion`: API Version for KMS provider. Set this to `v2`.
  * `name`: Display name of the KMS plugin. Cannot be changed once set.
  * `endpoint`: Listen address of the gRPC server (KMS plugin). The endpoing is a UNIX domain socket.
  * `timeout`: How long should `kube-apiserver` wait for kms-plugin to respond before returning an error (default is 3 seconds).

### **1-3.** KMS Plugin Implementation <a href="#id-31-kms-version" id="id-31-kms-version"></a>

The **KMS plugin** can be implemented in the following ways. The plugin must be deployed within the **Kubernetes control plane**.

* Use the method provided by the **Cloud Provider**.
* Develop the **KMS plugin** as a **gRPC server**.

**Key4C K8S Resource Security** is implemented as a **KMS plugin based on a gRPC server**.\
Therefore, it may not be compatible with certain cloud providers depending on their supported configurations.

***

## **2.** Interface

The interface definitions for each KMS version are provided in the official Kubernetes documentation.\
You can refer to the interface specifications for each version at the following links.

* **KMS v1**
  * Golang: <https://github.com/kubernetes/kms/blob/release-1.33/apis/v1beta1/api.pb.go>
  * Other Languages: <https://github.com/kubernetes/kms/blob/release-1.33/apis/v1beta1/api.proto>
* **KMS v2**
  * Golang: <https://github.com/kubernetes/kms/blob/release-1.33/apis/v2/api.pb.go>
  * Other Languages: <https://github.com/kubernetes/kms/blob/release-1.33/apis/v2/api.proto>

***

## **3.** Implementation Reference

You can verify the implementation through the following link.

* <https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/#encrypting-your-data-with-the-kms-provider>
* <https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/#verifying-that-the-data-is-encrypted>
* <https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/#switching-from-a-local-encryption-provider-to-the-kms-provider>

***

## **4.** How to Apply Key4C K8S Resource Security <a href="#id-6-key4c-k8s-resource-security" id="id-6-key4c-k8s-resource-security"></a>

**Key4C K8S Resource Security** (hereinafter referred to as **Key4C K8S App**) supports both versions of the Kubernetes KMS plugin.\
Follow the steps below to apply and configure the Key4C K8S App.

### **4-1.** Prerequisites <a href="#id-61" id="id-61"></a>

Before using the Key4C K8S App via the web console, verify the following two items

* `KMS_USER_ID`: The ID value of the token generated when the service is registered.
* `KMS_KEY_ID`: The ID value of the key created for encryption and decryption.

### **4-2.** Preparing and Running the Key4C K8S App <a href="#id-61" id="id-61"></a>

<mark style="color:red;">**(!) important**</mark>

Before running the Key4C K8S App, please note the following:

* The Key4C K8S App must reside on the same host as the **Control Plane**.
* The Kubernetes **API Server** must recognize the KMS plugin before it starts.\
  Therefore, it **must be created as a static pod**.
* It must be created as a **single pod**.

Refer to the following example and apply the corresponding **YAML file** in your Kubernetes environment.

> The `key4c-container-image` can be downloaded from the **Download** page.

```bash
# key4c-kms-plugin-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: key4c-kms-plugin
  namespace: kube-system # It must be in the same namespace as the API server.
spec:
  containers:
  - name: key4c-kms-plugin-encryption-provider
    image: {{key4c-container-image}}
    securityContext:
      privileged: true
    env:
    - name: KMS_USER_ID
      value: {{KMS_USER_ID}}
    - name: KMS_KEY_ID
      value: {{KMS_KEY_ID}}
    volumeMounts:
    - name: kms-plugin-socket
      mountPath: /tmp # The location of the Unix domain socket used for communication with Kubernetes.
    volumes:
    - name: kms-plugin-socket
      hostPath:
        type: DirectoryOrCreate
        path: /tmp
```

To deploy the Key4C K8S App as a static pod, add the YAML file to the Kubernetes manifest configuration.\
The manifest file must be located in `/etc/kubernetes/manifests/`.

```bash
cp key4c-kms-plugin-pod.yaml /etc/kubernetes/manifests
```

Once the pod has been successfully created, you can check its status using the `kubectl` command.

```bash
kubectl get pod -n kube-system
NAME                      READY STATUS  RESTARTS      AGE
......
key4c-kms-plugin-xxxxxx   1/1   Running 0             87s
......
```

After the container starts successfully, a Unix domain socket named **kms-plugin-socket** should be created in the `/tmp` directory. You can verify its presence to confirm proper initialization.

***

### **4-3.** Applying the Key4C KMS Plugin <a href="#id-61" id="id-61"></a>

Once the **key4c-kms-plugin pod** has been successfully deployed, configure the system so that **Kubernetes (K8S)** can communicate with the **KMS plugin** as follows.

**4-3-1.** Creating the Encryption Configuration File

Create the file `/etc/kubernetes/pki/encryption-configuration.yaml` with the following contents.\
The configuration format may vary depending on the version of Kubernetes.

{% hint style="success" %} <mark style="color:blue;">**Note**</mark>

The path `/etc/kubernetes/pki/encryption-configuration.yaml` is not mandatory.\
You can specify any file location as long as it is correctly referenced in the **kube-apiserver** configuration.
{% endhint %}

* **KMS plugin v1**
  * `endpoint`: The endpoint path must match the location of the **kms-plugin-socket** specified in `key4c-kms-plugin-pod.yaml`.
  * `cachesize`: This parameter is available only in v1 and specifies the DEK (Data Encryption Key) cache size.

```bash
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - kms:
    name: key4c-kms-plugin
    endpoint: unix:///tmp/kms.sock
    cachesize: 1000
    timeout: 3s
```

* **KMS plugin v2**
  * `apiVersion`: The API version must be explicitly specified.
  * `endpoint`: The endpoint path must match the location of the **kms-plugin-socket** specified in `key4c-kms-plugin-pod.yaml`.

```bash
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - kms:
    apiVersion: v2
    name: key4c-kms-plugin
    endpoint: unix:///tmp/kms.sock
    timeout: 3s
```

#### **4-3-2.** Updating the API Server Configuration

In the API server configuration file (`/etc/kubernetes/manifests/kube-apiserver.yaml`), specify the **encryption-provider** setting.\
You must also mount the directory containing the socket created by the **KMS plugin** so that the **API server** can access it.

{% hint style="warning" %} <mark style="color:orange;">**Warning**</mark>

Once you modify and save the `kube-apiserver.yaml` file, the API server will restart automatically.\
During this process, all Control Plane pods will be restarted, and the KMS plugin will be applied accordingly.
{% endhint %}

```bash
apiVersion: v1
kind: Pod
metadata:
  ...
spec:
  containers:
  - command:
    - kube-apiserver
    ...
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --encryption-provider-config=/etc/kubernetes/pki/encryption-configuration.yaml
...
volumeMounts:
- mountPath: /tmp
  name: kms-plugin-socket
...
volumes:
- hostPath:
    path: /tmp
    type: DirectoryOrCreate
  name: kms-plugin-socket
```

#### **4-3-3.** Verifying KMS Plugin Installation

Check that the following Kubernetes components are running properly:

* **kube-apiserver**
* **kube-controller-manager**
* **kube-scheduler**

```bash
kubectl get pod -n kube-system
NAME                            READY STATUS  RESTARTS      AGE
......
kube-apiserver-xxxxxx           1/1   Running 0             87s
kube-controller-manager-xxxxxx  1/1   Running 0             87s
kube-scheduler-xxxxxx           1/1   Running 0             87s
key4c-kms-plugin-xxxxxx         1/1   Running 0             10m
......
```

***

### **4-4.** Verifying the Key4C KMS Plugin <a href="#id-61" id="id-61"></a>

Create a test **Secret** to verify the plugin functionality.

```bash
kubectl create secret generic key4cSecret \
--from-literal=user=key4c \
--from-literal=pass=password
```

Create a test pod that loads the **Secret** as an environment variable.

```bash
# test-nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: test-nginx
  namespace: default
spec:
  containers:
  - name: test-nginx-container
    image: nginx:1.14
    env:
    - name: KEY4C_SECRET_USER
      valueFrom:
        secretKeyRef:
          name: key4cSecret
          key: user
    - name: KEY4C_SECRET_PASS
      valueFrom:
        secretKeyRef:
          name: key4cSecret
          key: pass
```

Run the pod.

```bash
kubectl create -f test-nginx-pod.yaml
```

Check inside the pod to ensure that the environment variable has been set correctly.

```bash
kubectl exec -it test-nginx -- /bin/bash
env | grep KEY4C
KEY4C_SECRET_USER=key4c
KEY4C_SECRET_PASS=password
```