Kyverno-Envoy-Plugin - Kyverno policies based authorization plugin for Envoy

Make external authorization easy with Kyverno-Envoy-Plugin.

By Sanskar Gurdasani | Tuesday, June 04, 2024

Microservices enhance the productivity of individual development teams by dividing applications into smaller, independent components. However, microservices alone do not address longstanding challenges in distributed systems such as authentication and authorization. These problems can become even harder to manage due to the diverse and short-lived nature of the microservice environments.

As more organizations move to using microservices, there is an increasing need for separate authentication and authorization mechanisms that work across different microservices.

In this blog post, we will introduce Kyverno-Envoy-Plugin, how it works and how you can use this version of Kyverno to enforce fine-grained, context-aware access control policies with Envoy without modifying your microservice or application code.

What is Envoy

Envoy is a Layer 7 proxy and communication bus tailored for large-scale, modern service-oriented architectures. Starting from version 1.7.0, Envoy includes an External Authorization filter that interfaces with an authorization service to check if the incoming request is authorized or not. This functionality allows authorization decisions to be offloaded to an external service, which can access the request context. The request context includes details such as the origin and destination of the network activity, as well as specifics of the network request (e.g., HTTP request). This information enables the external service to make a well-informed decision regarding the authorization of the incoming request processed by Envoy.

What is Kyverno-Envoy Plugin

Kyverno-Envoy plugin extends Kyverno-JSON with a gRPC server that implements Envoy External Authorization API. This allows you to enforce Kyverno policies on incoming and outgoing traffic in a service mesh environment, providing an additional layer of security and control over your applications. You can use this version of Kyverno to enforce fine-grained, context-aware access control policies with Envoy without modifying your microservice.

How does this work

In addition to the Envoy sidecar, your application pods will include a Kyverno-Envoy-Plugin component, either as a sidecar or as a separate pod. This Kyverno-Envoy-Plugin will be configured to communicate with the Kyverno-Envoy-Plugin gRPC server. When Envoy receives an API request intended for your microservice, it consults the Kyverno-Envoy-Plugin server to determine whether the request should be permitted.

Here is the architecture when Kyverno-Envoy-Plugin deployed as a sidecar to your application:

architecture-sidecar

Here is the architecture when Kyverno-Envoy-Plugin deployed as a separate pod to your application:

architecture-pod

Performing policy evaluations locally with Envoy is advantageous, as it eliminates the need for an additional network hop for authorization checks, thus enhancing both performance and availability.

The Kyverno-Envoy-Plugin can be deployed with Envoy-based sevice meshes such as Istio, Gloo, Kuma etc.

Getting started

In this blog, we will deploy the Kyverno-Envoy-Plugin as a sidecar container next to the application container. The plugin will handle authorizing incoming requests to the application. Additionally, documentation is provided for deploying the plugin as a separate pod.

Before we can look at Kyverno-Envoy-Plugin we need a Kubernetes cluster. We can create a local cluster with minikube and kubectl.

Create a local cluster

Start minikube cluster with the following command:

1minikube start

Install Kyverno-Envoy-Plugin sidecar with application

Install application with Envoy and Kyverno-Envoy-Plugin as a sidecar container.

1$ kubectl apply -f https://raw.githubusercontent.com/kyverno/kyverno-envoy-plugin/main/quick_start.yaml

The applicaition.yaml manifest defines the following resource:

The Deployment includes an example Go application that provides information of books in the library books collection and exposes APIs to get, create and delete books collection. Check this out for more information about the Go test application .
The Deployment also includes a Kyverno-Envoy-Plugin sidecar container in addition to the Envoy sidecar container. When Envoy recevies API request destined for the Go test applicaiton, it will check with Kyverno-Envoy-Plugin to decide if the request should be allowed and the Kyverno-Envoy-Plugin sidecar container is configured to query Kyverno-JSON engine for policy decisions on incoming requests.
A ConfigMap policy-config is used to pass the policy to Kyverno-Envoy-Plugin sidecar in the namespace default where the application is deployed .
A ConfigMap envoy-config is used to pass an Envoy configuration with an External Authorization Filter to direct authorization checks to the Kyverno-Envoy-Plugin sidecar.
The Deployment also includes an init container that install iptables rules to redirect all container traffic to the Envoy proxy sidecar container , more about init container can be found here

Make Test application accessible in the cluster

1kubectl expose deployment testapp --type=NodePort --name=testapp --port=8080

Set the `SERVICE_URL` environment variable to the service’s IP/port

minikube:

1export SERVICE_PORT=$(kubectl get service testapp -o jsonpath='{.spec.ports[?(@.port==8080)].nodePort}')
2export SERVICE_HOST=$(minikube ip)
3export SERVICE_URL=$SERVICE_HOST:$SERVICE_PORT
4echo $SERVICE_URL

Calling the sample test application and verify the authorization

For convenience, we’ll store Alice’s and Bob’s tokens in environment variables (not recommended for production). Here Bob is assigned the admin role and Alice is assigned the guest role.

1export ALICE_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjIyNDEwODE1MzksIm5iZiI6MTUxNDg1MTEzOSwicm9sZSI6Imd1ZXN0Iiwic3ViIjoiWVd4cFkyVT0ifQ.ja1bgvIt47393ba_WbSBm35NrUhdxM4mOVQN8iXz8lk"
2export BOB_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjIyNDEwODE1MzksIm5iZiI6MTUxNDg1MTEzOSwicm9sZSI6ImFkbWluIiwic3ViIjoiWVd4cFkyVT0ifQ.veMeVDYlulTdieeX-jxFZ_tCmqQ_K8rwx2OktUHv5Z0"

The policy we passed to Kyverno-Envoy-Plugin sidecar in the ConfigMap policy-config is configured to check the conditions of the incoming request and denies the request if the user is a guest and the request method is POST at the /book path.

 1apiVersion: json.kyverno.io/v1alpha1
 2kind: ValidatingPolicy
 3metadata:
 4  name: checkrequest
 5spec:
 6  rules:
 7    - name: deny-guest-request-at-post
 8      assert:
 9        any:
10        - message: "POST method calls at path /book are not allowed to guests users"
11          check:
12            request:
13                http:
14                    method: POST
15                    headers:
16                        authorization:
17                            (split(@, ' ')[1]):
18                                (jwt_decode(@ , 'secret').payload.role): admin
19                    path: /book                             
20        - message: "GET method call is allowed to both guest and admin users"
21          check:
22            request:
23                http:
24                    method: GET
25                    headers:
26                        authorization:
27                            (split(@, ' ')[1]):
28                                (jwt_decode(@ , 'secret').payload.role): admin
29                    path: /book 
30        - message: "GET method call is allowed to both guest and admin users"
31          check:
32            request:
33                http:
34                    method: GET
35                    headers:
36                        authorization:
37                            (split(@, ' ')[1]):
38                                (jwt_decode(@ , 'secret').payload.role): guest
39                    path: /book

Check for Alice which can get book but cannot create book.

1curl -i -H "Authorization: Bearer "$ALICE_TOKEN"" http://$SERVICE_URL/book

1curl -i -H "Authorization: Bearer "$ALICE_TOKEN"" -d '{"bookname":"Harry Potter", "author":"J.K. Rowling"}' -H "Content-Type: application/json" -X POST http://$SERVICE_URL/book

Check the Bob which can get book also create the book

1curl -i -H "Authorization: Bearer "$BOB_TOKEN"" http://$SERVICE_URL/book

1curl -i -H "Authorization: Bearer "$BOB_TOKEN"" -d '{"bookname":"Harry Potter", "author":"J.K. Rowling"}' -H "Content-Type: application/json" -X POST http://$SERVICE_URL/book

Check on logs

1kubectl logs "$(kubectl get pod -l app=testapp -o jsonpath={.items..metadata.name})" -c kyverno-envoy-plugin -f

First , third and last request is passed but second request is failed.

 1$ kubectl logs "$(kubectl get pod -l app=testapp -n demo -o jsonpath={.items..metadata.name})" -n demo -c kyverno-envoy-plugin -f
 2Starting HTTP server on Port 8000
 3Starting GRPC server on Port 9000
 4Request is initialized in kyvernojson engine .
 52024/04/26 17:11:42 Request passed the deny-guest-request-at-post policy rule.
 6Request is initialized in kyvernojson engine .
 72024/04/26 17:22:11 Request violation: -> POST method calls at path /book are not allowed to guests users
 8 -> any[0].check.request.http.headers.authorization.(split(@, ' ')[1]).(jwt_decode(@ , 'secret').payload.role): Invalid value: "guest": Expected value: "admin"
 9-> GET method call is allowed to both guest and admin users
10 -> any[1].check.request.http.headers.authorization.(split(@, ' ')[1]).(jwt_decode(@ , 'secret').payload.role): Invalid value: "guest": Expected value: "admin"
11 -> any[1].check.request.http.method: Invalid value: "POST": Expected value: "GET"
12-> GET method call is allowed to both guest and admin users
13 -> any[2].check.request.http.method: Invalid value: "POST": Expected value: "GET"
14Request is initialized in kyvernojson engine .
152024/04/26 17:23:13 Request passed the deny-guest-request-at-post policy rule.
16Request is initialized in kyvernojson engine .
172024/04/26 17:23:55 Request passed the deny-guest-request-at-post policy rule.

Configuration

To deploy Kyverno-Envoy-Plugin include the following container in your Kubernetes Deployments:

 1- name: kyverno-envoy-plugin
 2  image: sanskardevops/plugin:0.0.34
 3  imagePullPolicy: IfNotPresent
 4  ports:
 5    - containerPort: 8181
 6    - containerPort: 9000
 7  volumeMounts:
 8    - readOnly: true
 9  args:
10    - "serve"
11    - "--policy=/policies/policy.yaml"
12    - "--address=:9000"
13    - "--healthaddress=:8181"
14  livenessProbe:
15    httpGet:
16      path: /health
17      scheme: HTTP
18      port: 8181
19    initialDelaySeconds: 5
20    periodSeconds: 5
21  readinessProbe:
22    httpGet:
23      path: /health
24      scheme: HTTP
25      port: 8181
26    initialDelaySeconds: 5
27    periodSeconds: 5

Conclusion

This blog post demonstrates how the Kyverno-Envoy plugin can be effectively used to make external authorization decisions for incoming requests in a microservice architecture. By leveraging the power of Kyverno policies and Envoy’s External Authorization filter, you can achieve fine-grained, context-aware access control without modifying your application code. This approach not only simplifies the management of security policies but also enhances the security posture of your services by ensuring that authorization checks are consistently applied across your microservices.

The Kyverno-Envoy-Plugin provides a robust solution for organizations looking to enforce policy-driven access controls in their service meshes. By following the steps outlined in this guide, you can easily deploy and configure the plugin, allowing you to take full advantage of Kyverno’s policy capabilities in conjunction with Envoy’s powerful proxy features.

For further exploration, you can checkout: