Deploy Azure PostgreSQL Hyperscale on AKS using Azure DevOps Release Pipeline

The following README will guide you on how to use Azure DevOps (ADO) Release pipelines to deploy a “Ready to Go” environment so you can start using Azure Arc-enabled data services with Azure PostgreSQL Hyperscale (Citus) on Azure Kubernetes Service (AKS) cluster using Azure ARM Template.

By the end of this guide, you will have an Azure DevOps Release pipeline to deploy AKS cluster with an Azure Arc Data Controller (in “Directly Connected” mode, Azure PostgreSQL Hyperscale with a sample database and a Microsoft Windows Server 2022 (Datacenter) Azure VM, installed & pre-configured with all the required tools needed to work with Azure Arc Data Services.

Note: Currently, Azure Arc-enabled data services with PostgreSQL Hyperscale is in public preview.

Note: The following scenario is focusing the Azure DevOps Release pipeline creation. Once the pipeline has been created and the environment deployment has finished, the automation flow and next steps are as described on in the main bootstrap scenario

Prerequisites

  • Azure DevOps account set up with your organization and ready for project creation.

  • Install or update Azure CLI to version 2.15.0 and above. Use the below command to check your current installed version.

    az --version
    
  • Generate SSH Key (or use existing ssh key).

  • Create Azure service principal (SP)

    To be able to complete the scenario and its related automation, Azure service principal assigned with the “Contributor” role is required. To create it, login to your Azure account run the below command (this can also be done in Azure Cloud Shell).

    az login
    az ad sp create-for-rbac -n "<Unique SP Name>" --role contributor
    

    For example:

    az ad sp create-for-rbac -n "http://AzureArcData" --role contributor
    

    Output should look like this:

    {
    "appId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "displayName": "AzureArcData",
    "name": "http://AzureArcData",
    "password": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "tenant": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
    }
    

    Note: It is optional, but highly recommended, to scope the SP to a specific Azure subscription.

  • Enable subscription for the Microsoft.AzureArcData resource provider for Azure Arc-enabled data services. Registration is an asynchronous process, and registration may take approximately 10 minutes.

    az provider register --namespace Microsoft.AzureArcData
    

    You can monitor the registration process with the following commands:

    az provider show -n Microsoft.AzureArcData -o table
    

Deployment

In this scenario, you will create a new Release pipeline to deploy the environment ARM template for this Jumpstart scenario.

  • In a new or an existing ADO project, start the process of creating a new release pipeline.

    Screenshot of creating new ADO pipeline

    Screenshot of creating new ADO pipeline

  • To create the pipeline, we will be using an empty job template and give it a name (once done click the X button).

    Screenshot of creating new empty job template

    Screenshot of creating new empty job template

  • Create a new task for the stage you have just created. This task will be the one for deploying the ARM template.

    Screenshot of creating new ARM template deployment task

    Screenshot of creating new ARM template deployment task

  • Click on the new task to start it’s configuration.

    Screenshot of deployment task config

  • When deploying an ARM template, the Azure Resource Manager connection and subscription must be provided.

    Screenshot of Azure Resource Manager connection config

    Note: For new ADO project, you will be asked to click the authorization button

    Screenshot of Azure subscription config

  • Provide the Azure resource group and location where all the resources will be deployed. Make sure to validate if the service is currently available in your Azure region.

    Screenshot of resource group and location config

  • As mentioned, the task will deployed the existing ARM template for deploying Azure Arc-enabled data services with PostgreSQL Hyperscale that in the Azure Arc Jumpstart GitHub repository.

    • Change the Template location to “URL of the file”

    • Copy the raw URLs for both the template and the parameters json files and paste it in it’s the proper field.

    • The deployment ARM template requires you provide parameters values. Click on the Edit Override template parameters to add your parameters values.

      Screenshot of ARM template config

    Known issue: Currently, Azure Arc-enabled data services running on AKS with a Kubernetes version >=1.19.x is not supported. This is due to data services cannot be deployed on AKS cluster with availability zones enabled.

    • clusterName - AKS cluster name

    • dnsPrefix - AKS unique DNS prefix

    • nodeAdminUsername - AKS Node Username

    • sshRSAPublicKey - Your ssh public key

    • SPN_CLIENT_ID - Your Azure service principal name

    • SPN_CLIENT_SECRET - Your Azure service principal password

    • SPN_TENANT_ID - Azure tenant ID

    • ARC_DC_SUBSCRIPTION - Azure Arc Data Controller Azure subscription ID

    • ARC_DC_REGION - Azure location where the Azure Arc Data Controller resource will be created in Azure (Currently, supported regions supported are eastus, eastus2, centralus, westus2, westeurope, southeastasia)

    • kubernetesVersion - AKS Kubernetes Version (See previous prerequisite)

    • adminUsername - Client Windows VM admin username

    • adminPassword - Client Windows VM admin password

    • vmSize - Client Windows VM size

    • resourceGroup - Azure resource group where all the resources get deploy

    • AZDATA_USERNAME - Azure Arc Data Controller admin username

    • AZDATA_PASSWORD - Azure Arc Data Controller admin password (The password must be at least 8 characters long and contain characters from three of the following four sets: uppercase letters, lowercase letters, numbers, and symbols.)

    • ACCEPT_EULA - “yes” Do not change

    • ARC_DC_NAME - Azure Arc Data Controller name. The name must consist of lowercase alphanumeric characters or ‘-', and must start d end with a alphanumeric character (This name will be used for k8s namespace as well).

    • POSTGRES_NAME - PostgreSQL Hyperscale server group name to be deployed on the Kubernetes cluster. Names must be 10 characters or fewer in length and conform to DNS naming conventions.

    • POSTGRES_WORKER_NODE_COUNT - PostgreSQL Hyperscale server group number of workers

    • POSTGRES_DATASIZE - PostgreSQL Hyperscale size of data volumes in MB (Recommended to use at least 1GB (1024 MB)).*

    • POSTGRES_SERVICE_TYPE - Kubernetes service type i.e ClusterIP/LoadBalancer/NodePort. As AKS supports Load Balancers, leave configured with LoadBalancer.

      Note: Make sure that you are using the same Azure resource group name as the one you’ve just used in the azuredeploy.parameters.json file

      Screenshot of ARM template parameters config

      Screenshot of ARM template parameters config

      Screenshot of ARM template parameters config

      Screenshot of ARM template parameters config

  • Provide a deployment name.

    Screenshot of deployment name config

  • Click the save button.

    Screenshot of config save

  • After saving the task configuration, continue to create the release pipeline.

    Screenshot of pipeline creation

    Screenshot of pipeline creation

    Screenshot of pipeline creation

    Screenshot of pipeline creation

  • Once done, click on the new release link. In this scenario, you will perform a manually triggering for the deployment. Once you do, click on the Logs button to see the progress.

    Screenshot of pipeline deployment

    Screenshot of pipeline deployment

    Screenshot of deployment progress logs

    Screenshot of deployment progress logs

  • Once completed, all the deployment resources will be available in the Azure portal.

    Note: Deployment time of the Azure resources (AKS + Windows VM) can take ~25-30 minutes.

    Screenshot of deployment completed

    Screenshot of Azure resources

  • As mentioned, this scenario is focusing on the Azure DevOps Release pipeline creation. At this point, now that you have the Azure resources created, continue to the next steps as described on in the main bootstrap scenario.