Use Azure Policy on an Azure Arc-enabled Kubernetes cluster to audit pods’ labels for use with Calico network policy
Disclaimer: This scenario was contributed by the Tigera Project Calico team as part of the “Jumpstart Friends” program
The following Jumpstart scenario will guide you how to use Azure Policy Azure Policy for Kubernetes on an Azure Arc-enabled Kubernetes cluster to audit/enforce labels for pods. Network policy is applied to pods using label selectors.
NOTE: This guide assumes you already deployed an Amazon Elastic Kubernetes Service (EKS) or Google Kubernetes Engine (GKE) cluster and connected it to Azure Arc. If you haven’t, this repository offers you a way to do so in an automated fashion using these Jumpstart scenarios:
NOTE: This guide assumes you already have deployed Calico network policy in your cluster. If you haven’t, you can use our installation guides for Calico open source or Calico Cloud:
Calico Network Policy uses labels to select pods in Kubernetes for applying ingress/egress rules.
In this scenario, we will be using Azure Policy on an Azure-Arc enabled Kubernetes cluster to check whether the “fw-zone” label is present on pods in the “storefront” namespace, and how these labels are used in network policy enforcement.
The Azure Policy will be set to “Audit” mode to check the configuration of existing clusters (it can also be set to “Deny” mode to avoid any missing labels)
Clone the Azure Arc Jumpstart repository
git clone https://github.com/microsoft/azure_arc.git
Install or update Azure CLI to version 2.36.0 and above. Use the following command to check your current installed version.
As mentioned, this scenario starts at the point where you already have a connected EKS/GKE cluster to Azure Arc.
Before installing the Azure Policy Add-on or enabling any of the service features, your subscription must enable the Microsoft.PolicyInsights resource provider and create a role assignment for the cluster service principal. To do that, open Azure Cloud Shell and run either the Azure CLI or PowerShell command.
az provider register --namespace 'Microsoft.PolicyInsights'
Register-AzResourceProvider -ProviderNamespace 'Microsoft.PolicyInsights'
To verify successful registration, run either the below Azure CLI or PowerShell command.
az provider show --namespace 'Microsoft.PolicyInsights'
Get-AzResourceProvider -ProviderNamespace 'Microsoft.PolicyInsights'
Check whether Azure Policy is installed in an Azure Arc-enabled Kubernetes cluster
by running the
kubectl get pods -n gatekeeper-systemcommand.
You should see the following pods running.
Deploy a demo application and Calico Network policy in your Azure Arc-enabled Kubernetes cluster.
kubectl create -f storefront.yaml
storefrontdemo application in your cluster by running the
kubectl create -f storefront.yamlcommand.
kubectl create -f calico-firewall-policy.yaml
Deploy a sample Calico network policy by running the
kubectl create -f calico-firewall-policy.yamlcommand.
Deploy an Azure policy to verify if “fw-zone” label exists for pods in the
In the Azure portal search bar, look for Policy and click on Definitions which will show you all of the available Azure policies.
Click on Category to search for built-in policies.
In the below example, make sure Category is set to Kubernetes only, and type
labelin Search, you will find
Kubernetes cluster pods should use specified labelsin BuiltIn Type.
Click this policy, and assign it to your Azure resource group which includes the Azure Arc-enabled Kubernetes clusters. Alternatively, you can assign the policy to the entire Azure subscription.
Name the assignment, for example
storefront namespace pods should use fw-zone labels for calico _networkpolicy_ enforcement.
In the below example, we only audit one namespace which is
fw-zoneas the required label.
Once assigned, the policy task will start the evaluation against the Arc connected cluster in your resource group. You can check the status of this assignment under
NOTE: The process of evaluation can take up 30min, in the meantime will see 0 out of 0 in
After the evaluation, the policy will inform you of the result and provide details if there are any resources that are
We will deploy a centos and nginx pod to the
storefrontnamespace without the
fw-zonelabel by running the commands below. Creation of these pods will be allowed as the policy effect is set to
kubectl run centos --image=centos -n storefront --restart=Never -- /bin/sh -c "sleep 24h"
kubectl run nginx --image=nginx -n storefront kubectl -n storefront expose pod nginx --port=80
Test Calico network policy enforcement by adding the
fw-zonelabels to the
The network policy is set to allow only
trustedzone connections to the
restrictedzone, and will deny traffic from
dmzzone. We will label the nginx pod with
fw-zone=restricted, and test connectivity when changing the centos pod label from
Test Calico network policy enforcement without labels.
Test Calico network policy enforcement with proper labels.
Test Calico network policy enforcement with wrong labels.
Clean up environment
Complete the following steps to clean up your environment.
From each Azure Arc-enabled Kubernetes cluster under your resource group, click the Azure Policy page, and disable the extension of Azure policy under
Onboard to Azure Policy for Azure Arc-enabled Kubernetes cluster
You can use these Jumpstart scenarios below for deleting the clusters: