Deploy GKE cluster and connect it to Azure Arc using Terraform

The following README will guide you on how to use the provided Terraform plan to deploy a Google Cloud Platform Kubernetes Engine cluster and connected it as an Azure Arc cluster resource.

Prerequisites

  • Clone the Azure Arc Jumpstart repository

    git clone https://github.com/microsoft/azure_arc.git
    
  • Install or update Azure CLI to version 2.15.0 and above. Use the below command to check your current installed version.

    az --version
    
  • Create a Google Cloud account

  • Install Terraform >=0.12

  • Create Azure service principal (SP)

    To be able to complete the scenario and its related automation, Azure service principal assigned with the “Contributor” role is required. To create it, login to your Azure account run the below command (this can also be done in Azure Cloud Shell).

    az login
    az ad sp create-for-rbac -n "<Unique SP Name>" --role contributor
    

    For example:

    az ad sp create-for-rbac -n "http://AzureArcK8s" --role contributor
    

    Output should look like this:

    {
    "appId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "displayName": "AzureArcK8s",
    "name": "http://AzureArcK8s",
    "password": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "tenant": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }
    

    Note: The Jumpstart scenarios are designed with as much ease of use in-mind and adhering to security-related best practices whenever possible. It is optional but highly recommended to scope the service principal to a specific Azure subscription and resource group as well considering using a less privileged service principal account

  • Enable subscription with the two resource providers for Azure Arc enabled Kubernetes. Registration is an asynchronous process, and registration may take approximately 10 minutes.

    az provider register --namespace Microsoft.Kubernetes
    az provider register --namespace Microsoft.KubernetesConfiguration
    

    You can monitor the registration process with the following commands:

    az provider show -n Microsoft.Kubernetes -o table
    az provider show -n Microsoft.KubernetesConfiguration -o table
    
  • Install the Azure Arc for Kubernetes CLI extensions connectedk8s and k8s-configuration:

    az extension add --name connectedk8s
    az extension add --name k8s-configuration
    

    Note: If you already used this guide before and/or have the extensions installed, use the bellow commands:

    az extension update --name connectedk8s
    az extension update --name k8s-configuration
    

Create a new GCP Project

  • Browse to https://console.cloud.google.com/ and login with your Google Cloud account. Once logged in, create a new project named “Azure Arc Demo”. After creating it, be sure to copy down the project id as it is usually different then the project name.

    GCP new project

    GCP new project

    GCP new project

  • Enable the Compute Engine API for the project, create a project Owner service account credentials and download the private key JSON file and copy the file to the directory where Terraform files are located. Change the JSON file name (for example account.json). The Terraform plan will be using the credentials stored in this file to authenticate against your GCP project.

    Enable Compute Engine API

    Enable Compute Engine API

    Add credentials

    Add credentials

    Add credentials

    Add credentials

    Add credentials

    Create private key

    Create private key

    Create private key

    Create private key

    account.json

  • Enable the Kubernetes Engine API for the project

    Enable the Kubernetes Engine API

    Enable the Kubernetes Engine API

Deployment

The only thing you need to do before executing the Terraform plan is to export the environment variables which will be used by the plan. This is based on the Azure service principal you’ve just created and your subscription.

  • Export the environment variables needed for the Terraform plan.

    export TF_VAR_subscriptionId='<Your Azure subscription ID>'
    export TF_VAR_servicePrincipalAppId='<Your Azure service principal App ID>'
    export TF_VAR_servicePrincipalSecret='<Your Azure service principal App Password>'
    export TF_VAR_servicePrincipalTenantId='<Your Azure tenant ID'
    export TF_VAR_gcp_project_id='<Your GCP Project ID>'
    export TF_VAR_location='<Azure Region>'
    export TF_VAR_resourceGroup='<Azure resource group name>'
    export TF_VAR_gcp_credentials_filename='<Location on the Keys JSON file>'
    export TF_VAR_gcp_region='<GCP Region to deploy resources>'
    export TF_VAR_gke_cluster_name='<GKE cluster name>'
    export TF_VAR_admin_username='<GKE control plane administrator username>'
    export TF_VAR_admin_password='<GKE control plane administrator password>'
    export TF_VAR_gke_cluster_node_count='<GKE cluster node count>'
    export TF_VAR_gke_cluster_node_machine_type='<GKE cluster node machine type>'
    

    For example:

    export TF_VAR_subscriptionId='XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
    export TF_VAR_servicePrincipalAppId='XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
    export TF_VAR_servicePrincipalSecret='XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
    export TF_VAR_servicePrincipalTenantId='XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX'
    export TF_VAR_location='eastus'
    export TF_VAR_resourceGroup='Arc-GKE-Demo'
    export TF_VAR_gcp_project_id='azure-arc-demo-111111'
    export TF_VAR_gcp_credentials_filename='account.json'
    export TF_VAR_gcp_region='us-west1'
    export TF_VAR_gke_cluster_name='arc-gke-demo'
    export TF_VAR_admin_username='arcdemo'
    export TF_VAR_admin_password='ArcDemo1234567!!'
    export TF_VAR_gke_cluster_node_count=1
    export TF_VAR_gke_cluster_node_machine_type='n1-standard-2'
    

    Note: If you are running in a PowerShell environment, to set the Terraform environment variables, use the Set-Item -Path env: prefix (see example below)

    Set-Item -Path env:TF_VAR_gcp_project_id
    
  • Run the terraform init command which will download the required terraform providers.

    terraform init

  • Run the terraform apply --auto-approve command and wait for the plan to finish. Once done, you will have a new empty Azure resource group and and a GKE cluster under the Kubernetes Engine page in your GCP console.

    terraform apply

    An empty Azure resource group

    New GKE cluster in the Google Console

    New GKE cluster in the Google Console

Connecting to Azure Arc

  • Now that you have a running GKE cluster, retrieve your Azure subscription ID using the az account list command and edit the environment variables section in the included az_connect_gke shell script.

    Export environment variables

  • Open a new Cloud Shell session which will pre-authenticated against your GKE cluster.

    Open Google Cloud Shell session and authenticate against the GKE cluster

    Open Google Cloud Shell session and authenticate against the GKE cluster

    Open Google Cloud Shell session and authenticate against the GKE cluster

  • Upload the az_connect_gke shell script and run it using the . ./az_connect_gke.sh command.

    Note: The extra dot is due to the script has an export function and needs to have the vars exported in the same shell session as the rest of the commands.

    Upload a file to Cloud Shell

    Upload a file to Cloud Shell

    Upload a file to Cloud Shell

  • Upon completion, you will have your GKE cluster connect as a new Azure Arc Kubernetes cluster resource in the new Azure resource group.

    New Azure Arc enabled Kubernetes cluster

    New Azure Arc enabled Kubernetes cluster

    New Azure Arc enabled Kubernetes cluster

Delete the deployment

To delete the environment, use the terraform destroy --auto-approve command.

terraform destroy