Deploy Rancher k3s on a VMware vSphere VM and connect it to Azure Arc using Terraform

The following README will guide you on how to use the provided Terraform plan to deploy a “Ready to Go” VMware vSphere Ubuntu Server virtual machine installed with a single-master Rancher K3s Kubernetes cluster and connected it as an Azure Arc cluster resource.

Prerequisites

  • Clone the Azure Arc Jumpstart repository

    git clone https://github.com/microsoft/azure_arc.git
    
  • Install or update Azure CLI to version 2.15.0 and above. Use the below command to check your current installed version.

    az --version
    
  • Install Terraform >=0.12

  • A VMware vCenter Server user with permissions to deploy a Virtual Machine from a Template in the vSphere Web Client.

  • Create Azure service principal (SP)

    To be able to complete the scenario and its related automation, Azure service principal assigned with the “Contributor” role is required. To create it, login to your Azure account run the below command (this can also be done in Azure Cloud Shell).

    az login
    az ad sp create-for-rbac -n "<Unique SP Name>" --role contributor
    

    For example:

    az ad sp create-for-rbac -n "http://AzureArcK8s" --role contributor
    

    Output should look like this:

    {
    "appId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "displayName": "AzureArcK8s",
    "name": "http://AzureArcK8s",
    "password": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "tenant": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }
    

    Note: The Jumpstart scenarios are designed with as much ease of use in-mind and adhering to security-related best practices whenever possible. It is optional but highly recommended to scope the service principal to a specific Azure subscription and resource group as well considering using a less privileged service principal account

  • Enable subscription with the two resource providers for Azure Arc enabled Kubernetes. Registration is an asynchronous process, and registration may take approximately 10 minutes.

    az provider register --namespace Microsoft.Kubernetes
    az provider register --namespace Microsoft.KubernetesConfiguration
    

    You can monitor the registration process with the following commands:

    az provider show -n Microsoft.Kubernetes -o table
    az provider show -n Microsoft.KubernetesConfiguration -o table
    

Preparing an Ubuntu Server VMware vSphere VM Template

Before using the below guide to deploy an Ubuntu Server VM and connect it to Azure Arc, a VMware vSphere Template is required. The following README will instruct you how to easily create such a template using VMware vSphere 6.5 and above.

Note: If you already have an Ubuntu Server VM template it is still recommended to use the guide as a reference.

Deployment

Before executing the Terraform plan, you must set the environment variables which will be used by the plan. These variables are based on the Azure service principal you’ve just created, your Azure subscription and tenant, and your VMware vSphere credentials.

  • Retrieve your Azure subscription ID and tenant ID using the az account list command.

  • The Terraform plan creates resources in both Microsoft Azure and VMware vSphere. It then executes a script on the virtual machine to install the Azure Arc agent and all necessary artifacts. This script requires certain information about your VMware vSphere and Azure environments.

  • Edit scripts/vars.sh and update each of the variables with the appropriate values.

    • TF_VAR_subscription_id=Your Azure subscription ID
    • TF_VAR_client_id=Your Azure service principal name
    • TF_VAR_client_secret=Your Azure service principal password
    • TF_VAR_tenant_id=Your Azure tenant ID
    • TF_VAR_resourceGroup=Azure resource group name
    • TF_VAR_location=Azure Region
    • TF_VAR_arcClusterName=The name of your k8s cluster as it will be shown in Azure Arc
    • TF_VAR_vsphere_user=vCenter Admin Username
    • TF_VAR_vsphere_password=vCenter Admin Password
    • TF_VAR_vsphere_server=vCenter server FQDN/IP
    • TF_VAR_admin_user=OS Admin Username
    • TF_VAR_admin_password=OS Admin Password
  • From CLI, navigate to the azure_arc_k8s_jumpstart/rancher_k3s/vmware/terraform directory of the cloned repo.

  • Export the environment variables you edited by running scripts/vars.sh with the source command as shown below. Terraform requires these to be set for the plan to execute properly. Note that this script will also be automatically executed remotely on the virtual machine as part of the Terraform deployment.

    source ./scripts/vars.sh
    
  • In addition to the TF_VAR environment variables you’ve just exported, edit the Terraform variables in the terraform.tfvars to match your VMware vSphere environment.

    TF_VAR environment variables

  • Run the terraform init command which will download the Terraform AzureRM, Local and vSphere providers.

    terraform init

  • Run the terraform apply --auto-approve command and wait for the plan to finish.

    terraform apply

  • Once the Terraform deployment is completed, a new Ubuntu Server VM will be up & running, installed with a single-master Rancher K3s Kubernetes cluster and will be projected as an Azure Arc Kubernetes cluster in a newly created Azure resource group.

    VMware vSphere Ubuntu Server VM

    Azure Arc enabled Kubernetes cluster

    Azure Arc enabled Kubernetes cluster

Delete the deployment

  • The most straightforward way is to delete the cluster is via the Azure Portal, just select cluster and delete it.

    Delete Azure Arc enabled Kubernetes cluster

  • If you want to nuke the entire environment, just delete the Azure resource group and the newly generated az_connect_k3s shell script in the scripts folder.

    Delete Azure resource group

  • Alternatively, you can use the terraform destroy --auto-approve command.

    terraform destroy