Deploy Rancher k3s on an Azure VM and connect it to Azure Arc using Terraform

The following README will guide you on how to use the provided Terraform plan to deploy a “Ready to Go” Azure virtual machine installed with single-master Rancher K3s Kubernetes cluster and connected it as an Azure Arc cluster resource.

Prerequisites

  • Clone the Azure Arc Jumpstart repository

    git clone https://github.com/microsoft/azure_arc.git
    
  • Install or update Azure CLI to version 2.15.0 and above. Use the below command to check your current installed version.

    az --version
    
  • Install Terraform >=0.12

  • Create Azure service principal (SP)

    To be able to complete the scenario and its related automation, Azure service principal assigned with the “Contributor” role is required. To create it, login to your Azure account run the below command (this can also be done in Azure Cloud Shell).

    az login
    az ad sp create-for-rbac -n "<Unique SP Name>" --role contributor
    

    For example:

    az ad sp create-for-rbac -n "http://AzureArcK8s" --role contributor
    

    Output should look like this:

    {
    "appId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "displayName": "AzureArcK8s",
    "name": "http://AzureArcK8s",
    "password": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "tenant": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }
    

    Note: The Jumpstart scenarios are designed with as much ease of use in-mind and adhering to security-related best practices whenever possible. It is optional but highly recommended to scope the service principal to a specific Azure subscription and resource group as well considering using a less privileged service principal account

  • Enable subscription with the two resource providers for Azure Arc enabled Kubernetes. Registration is an asynchronous process, and registration may take approximately 10 minutes.

    az provider register --namespace Microsoft.Kubernetes
    az provider register --namespace Microsoft.KubernetesConfiguration
    

    You can monitor the registration process with the following commands:

    az provider show -n Microsoft.Kubernetes -o table
    az provider show -n Microsoft.KubernetesConfiguration -o table
    
  • The Terraform plan execute a script on the VM OS to install all the needed artifacts as well to inject environment variables. Edit the scripts/vars.sh to match the Azure service principal you’ve just created.

Deployment

The only thing you need to do before executing the Terraform plan is to export the environment variables which will be used by the plan. This is based on the Azure service principal you’ve just created and your subscription.

  • Retrieve your Azure subscription ID using the az account list command.

  • Export the environment variables needed for the Terraform plan.

    export TF_VAR_subscription_id=<Your Azure subscription ID>  
    export TF_VAR_client_id=<Your Azure service principal App ID>
    export TF_VAR_client_secret=<Your Azure service principal App password>  
    export TF_VAR_tenant_id=<Your Azure service principal Tenant ID>
    

    Note: If you are running in a PowerShell environment, to set the Terraform environment variables, use the Set-Item -Path env: prefix (see example below)

    Set-Item -Path env:TF_VAR_subscription_id
    
  • Run the terraform init command which will download the Terraform AzureRM provider.

    terraform init

  • Run the terraform apply --auto-approve command and wait for the plan to finish.

    terraform apply completed

Connecting to Azure Arc

Note: The VM bootstrap includes the log in process to Azure as well deploying the needed Azure Arc CLI extensions - no action items on you there!

  • SSH to the VM using the created Azure Public IP and your username/password.

    Azure VM public IP

  • Check the cluster is up and running using the kubectl get nodes -o wide

    k3s cluster nodes

  • Using the Azure service principal you’ve created, run the below command to connect the cluster to Azure Arc.

    az connectedk8s connect --name <Name of your cluster as it will be shown in Azure> --resource-group <Azure resource group name>
    

    For example:

    az connectedk8s connect --name arck3sdemo --resource-group Arc-K3s-Demo
    

    Successful azconnctedk8s command

    Azure Arc enabled Kubernetes cluster in an Azure resource group

    Azure Arc enabled Kubernetes cluster in an Azure resource group

K3s External Access

Traefik is the (default) ingress controller for k3s and uses port 80. To test external access to k3s cluster, an “hello-world” deployment was made available for you and it is included in the home directory (credit).

  • Since port 80 is taken by Traefik (read more about here), the deployment LoadBalancer was changed to use port 32323 along side with the matching Azure Network Security Group (NSG).

    Azure Network Security Group (NSG) rule

    hello-kubernetes.yaml file

  • To deploy it, use the kubectl apply -f hello-kubernetes.yaml command. Run kubectl get pods and kubectl get svc to check that the pods and the service has been created.

    kubectl apply -f hello-kubernetes.yaml command

    kubectl get pods command

    kubectl get svc command

  • In your browser, enter the cluster_public_ip:3232 which will bring up the hello-world application.

    hello-kubernetes application in a web browser

Delete the deployment

  • The most straightforward way is to delete the cluster is via the Azure Portal, just select cluster and delete it.

    Delete Azure Arc enabled Kubernetes cluster

  • If you want to nuke the entire environment, just delete the Azure resource group or alternatively, you can use the terraform destroy --auto-approve command.

    Delete Azure resource group

    terraform destroy