Last updated on March 28th, 2024 at 12:20 pm
In this tutorial I will walk you through 4 simple steps for deploying a Validating webhook in Kubernetes cluster and an example of adding validation based on your use case.
Before we begin make sure that you have a running Kubernetes cluster, kubectl should be installed to manage that cluster. One other command line tool required is openssl.
An Admission controller can be validating, mutating, or both. Mutating controllers may modify objects related to the requests they admit; validating controllers may not.
In this example I will be creating a Validating controller. More details can be found in these documents
- https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
- https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
One of the cool feature of webhooks is that you can write your own logic on how an object for example a deployment needs to be created, whether it has to have a specific label or allow those deployments to be created on a given day of the week etc., . You can develop these logic in any programming language (python, php, go etc.,) as long as you can catch a post request and create a JSON response to allow or forbid the request
For example, below JSON response from your webhook allow the request to go through, since allowed key is having value true. You can capture the UID of a specific request from the POST payload.
{
"apiVersion": "admission.k8s.io/v1",
"kind": "AdmissionReview",
"response": {
"uid": "<value from request.uid>",
"allowed": true
}
}
- Step 1 Create Self Signed Certificate
- Step 2 Deploy docker image
- Step 3 Configure Kubernetes cluster with Webhook
- Step 4 Validate Webhook
A sample request POST payload looks like this, (truncated) – you can see the uid inside request block. This gets triggered when someone try to create a new object like Deployment / PODS – depending on the resources you provided as part of the rules (inside ValidatingWebhookConfiguration)
{"kind":"AdmissionReview","apiVersion":"admission.k8s.io\/v1","request":{"uid":"d7b51f8b-648d-42e9-9bbc-6aa8dad9157e","kind":{"group":"apps","version":"v1","kind":"Deployment"},"resource":{"group":"apps","version":"v1","resource":"deployments"},......
Without further delay lets jump on to the setup
Step 1 Create Self Signed Certificate
Using openssl we are going to create a self signed certificate. If you have requirement to use your own CA signed certificates, use them accordingly. Here I am going to create certificates for my service endpoint named “mywebhook.default.svc” (replace that with your own service name – if required)
$ openssl req -x509 -sha256 -newkey rsa:2048 -keyout server.key -out server.crt -days 1024 -nodes -addext "subjectAltName = DNS.1:mywebhook.default.svc"
Generating a 2048 bit RSA private key
.....+++++
.......................................................................+++++
writing new private key to 'server.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) []:US
State or Province Name (full name) []:
Locality Name (eg, city) []:
Organization Name (eg, company) []:
Organizational Unit Name (eg, section) []:
Common Name (eg, fully qualified host name) []:mywebhook.default.svc
Email Address []:
$
Now that the SSL certificates are created, lets verify the cert (truncated)
$ openssl x509 -in server.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
a0:ff:0f:28:1e:fe:66:42
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=US, CN=mywebhook.default.svc
Validity
Not Before: Mar 21 17:02:38 2024 GMT
Not After : Jan 9 17:02:38 2027 GMT
Subject: C=US, CN=mywebhook.default.svc
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:b7:0c:cd:07:e5:be:87:ef:5a:df:12:2f:ae:32:...
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Alternative Name:
DNS:mywebhook.default.svc
Signature Algorithm: sha256WithRSAEncryption
64:9e:14:2a:8e:e7:3a:78:a4:56:11:79:14:1e:df:7e:24:e8:....
We now have the certs ready.
Step 2 Deploy docker image
For the benefit of this guide I am using my own custom repo. You are free to use your own.
Let us create a PHP code to accept the POST request and process the data. We are extracting the uid and label from the payload then accordingly create a response_data. Write it to application log and also echo the json payload for the webhook to process it.
For example, in your deployment manifest if there is no label with key as environment and value as stage the webhook will reject the request. We will see more details with sample output in the step 4 section.
We are also parsing the POST request and writing that to a log file. There will be 2 log entries, first one will show the response along with the label extracted and the second is just a dump of post request from kind AdmissionReview. Adding logs will enable us to easily debug and trace the issue if there are any errors while triggering the webhook.
<?php
$raw_payload = file_get_contents('php://input', true);
$payload = json_decode($raw_payload, true);
$json=json_encode($payload);
$uid = $payload['request']['uid'];
$label = $payload['request']['object']['metadata']['labels'];
if ($label['environment'] == 'stage')
{
$validate=true;
$code=200;
$message="Webhook validation passed";
}
else
{
$validate=false;
$code=403;
$message="You cannot do this because the environment is not tagged as stage. Current value is ".$label['environment'];
}
$response_data = [
"apiVersion" => "admission.k8s.io/v1",
"kind" => "AdmissionReview",
"response" => [
"uid" => $uid,
"allowed" => $validate,
"status" => [
"code" => $code,
"message" => $message
]
]
];
$response_data_from_app= json_encode($response_data);
$log = date("Y-m-d h:i:sa")." - ".$uid." - ".$response_data_from_app." - ".$label['environment']."\n";
$original = date("Y-m-d h:i:sa")." - ".$uid." - ".$json."\n";
file_put_contents('./log_'.date("j.n.Y").'.log', $log, FILE_APPEND);
file_put_contents('./log_'.date("j.n.Y").'.log', $original, FILE_APPEND);
$json= json_encode($response_data);
echo header("Content-Type: application/json; charset=utf-8");
echo stripslashes(trim($json));
?>
My Dockerfile
# Derived from official PHP image (our base image)
FROM php:7.2-apache
COPY server.key /etc/ssl/private/
COPY server.crt /etc/ssl/certs/
COPY index.php /var/www/html
RUN sed -i s/ssl-cert-snakeoil.key/server.key/ /etc/apache2/sites-available/default-ssl.conf
RUN sed -i s/ssl-cert-snakeoil.pem/server.crt/ /etc/apache2/sites-available/default-ssl.conf
RUN a2enmod ssl
RUN a2ensite default-ssl
All I am doing here is enabling SSL for Apache and copying the certs we created in Step 1.
We are all set, build it and push it in your own repository. I am using Docker to save my image.
Step 3 Configure Kubernetes cluster with Webhook
Now that we have the image set up for the webhook. Since my image resides in private repo I have to create a secret for use with Docker registry (modify the values accordingly if you are using private repo)
$ kubectl create secret docker-registry my-registry-secret--docker-server=https://index.docker.io/v1/ --docker-username=xxx--docker-password=xxx [email protected]
Now lets create a deployment using the below manifest, name it deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: validation-webhook
labels:
app: validate
spec:
replicas: 1
selector:
matchLabels:
app: validate
template:
metadata:
labels:
app: validate
spec:
containers:
- name: webhook
image: xxx/mytestproject:apache-php-webhook-working
ports:
- containerPort: 443
imagePullSecrets:
- name: my-registry-secret
Lets create a service from the deployment above – service.yaml, all we are doing here is selecting the pod with the label app=validate. The port we are connecting is 443
$ cat service.yaml
apiVersion: v1
kind: Service
metadata:
name: mywebhook
spec:
selector:
app: validate
ports:
- port: 443
$
We now have a Webhook reachable at https://mywebhook.default.svc
Next step is to have the webhook.yaml deployed for ValidatingWebhookConfiguration kind.
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: validating-webhook
webhooks:
- name: mywebhook.default.svc
failurePolicy: Fail
sideEffects: None
admissionReviewVersions: ["v1","v1beta1"]
rules:
- apiGroups: ["apps", ""]
resources:
- "deployments"
apiVersions: [ "v1" ]
operations:
- CREATE
clientConfig:
service:
name: mywebhook
namespace: default
path: /
caBundle: BASE64 ENCODED SERVER.CRT DATA
Execute this command to get the BASE64 ENCODED SERVER.CRT DATA encoded value of the server.crt file (truncated).
$ cat server.crt |base64 | tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURBVENDQWVtZ0F3SUJBZ0lKQUtEL0R5Z2Uv
bVpDTUEwR0NTcUdTSWIzRFFFQkN3VUFNQzB4Q3pBSkJnTlYKQkFZVEFsVlRNUjR3SEFZRFZRUURE
QlZ0ZVhkbFltaHZiMnN1WkdWbVlYVnNkQzV6ZG1Nd0hoY05NalF3TXpJeApNVGN3TWpNNFdoY05N
amN3TVRBNU1UY3dNak00V2pBdE1Rc3dDUVlEVlFRR0V3SlZVekVlTUJ3R0ExVUVBd3dWCmJYbDNa
V0pvYjI5ckxtUmxabUYxYkhRdWMzWmpNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1J
In the example above, we intercept DEPLOYMENT objects and CREATE operations – all new deployments that are going to be created in the Kubernetes cluster.
Once the above manifests are ready run these commands
$ kubectl create -f deployment.yaml
$ kubectl create -f webhook.yaml
Step 4 Validate Webhook
For the validation test I have a deployment manifest file – lets call it test_nginx_deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: nginx
environment: dev
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
resources: {}
status: {}
In the example above environment is labeled as dev and not stage. So if someone deploy the above test_nginx_deployment.yaml, our validation webhook should prevent the creation of this deployment, since it fails the label check.
$ kubectl apply -f test_nginx_deployment.yaml
Error from server: error when creating "test_nginx_deployment.yaml": admission webhook "mywebhook.default.svc" denied the request: You cannot do this because the environment is not tagged as stage. Current value is dev
$
Update the label from dev to stage and try deployment again. If that goes through then our webhook validation is working as expected.
Sample application logs (truncated – I have added the validation fail and successful logs). As you can see it is correctly hitting the service endpoint.
#After changing the label from dev to stage the deployment was successful
$ kubectl apply -f test_nginx_deployment.yaml
deployment.apps/nginx created
#Lets check the logs for today
$ kubectl exec -it validation-webhook-68fc59f6f5-8txms -- cat /var/www/html/log_22.3.2024.log
#Validation failed due to wrong label
2024-03-22 06:40:41pm - d8fc22f8-18ba-454a-852b-01985d9c5857 - {"apiVersion":"admission.k8s.io\/v1","kind":"AdmissionReview","response":{"uid":"d8fc22f8-18ba-454a-852b-01985d9c5857","allowed":false,"status":{"code":403,"message":"You cannot do this because the environment is not tagged as stage. Current value is stage1"}}} - dev
2024-03-22 06:40:41pm - d8fc22f8-18ba-454a-852b-01985d9c5857 - {"kind":"AdmissionReview","apiVersion":"admission.k8s.io\/v1","request":{"uid":"d8fc22f8-18ba-454a-852b-01985d9c5857","kind":{"group":"apps","version":"v1","kind":"Deployment"},"resource":{"group":"apps","version":"v1","resource":"deployments"},"requestKind":{"group":"apps","version":"v1","kind":"Deployment"},"requestResource":{"group":"apps","version":"v1","resource":"deployments"},"name":"nginx","namespace":"default","operation":"CREATE","userInfo":{"username":"kubernetes-admin","uid":"aws-iam-authenticator:xxxx:AROLHCAOQ","creationTimestamp":"2024-03-22T18:40:41Z","labels":{"app":"nginx","environment":"dev"}
#Validation successful, label found
2024-03-22 07:12:32pm - 07ed1324-7c1c-4148-b4c0-860bf84b068c - {"apiVersion":"admission.k8s.io\/v1","kind":"AdmissionReview","response":{"uid":"07ed1324-7c1c-4148-b4c0-860bf84b068c","allowed":true,"status":{"code":200,"message":"Webhook validation passed"}}} - stage
2024-03-22 07:12:32pm - 07ed1324-7c1c-4148-b4c0-860bf84b068c - {"kind":"AdmissionReview","apiVersion":"admission.k8s.io\/v1","request":{"uid":"07ed1324-7c1c-4148-b4c0-860bf84b068c","kind":{"group":"apps","version":"v1","kind":"Deployment"},"resource":{"group":"apps","version":"v1","resource":"deployments"},"requestKind":{"group":"apps","version":"v1","kind":"Deployment"},"requestResource":{"group":"apps","version":"v1","resource":"deployments"},"name":"nginx","namespace":"default","operation":"CREATE","userInfo":{"username":"kubernetes-admin","uid":"aws-iam-authenticator:xxxx:AROLHCAOQ","creationTimestamp":"2024-03-22T19:12:32Z","labels":{"app":"nginx","environment":"stage"}
Thanks for your time. Please let me know if you have any questions.
admin