Using K8GB on Azure

Sample solution

Sample solution will create a common hub and spoke architecture with two AKS clusters in different regions

Azure based deployment with Windows DNS integration

Here we provide an example of k8gb deployment in Azure environment with Windows DNS as edgeDNS provider.

Reference Setup

The reference setup includes two AKS clusters that can deployed on two different regions for load balancing or provide a failover solution.

The solution design can be found here.

Configurable resources:

• Resource groups
• VNet and subnets
• Peering
• Managed Identity
• Clusters

Run the sample

• This lab requires a running AD Domain Controller with DNS and KDC services working

  • There are several tutorials available online, but this Microsoft Learn article will probably help you out
  • Microsoft Learn

• To run the provided sample, please use the provided Makefile here.

  • Deploys all the required infrastructure and configurations
  • Before execute, please fill all the local variables in the scripts with the correct naming for the resources in order to avoid having problems with your azure policies
  • Scripts will use AZ CLI, ensure that is installed and logged when trying to execute the command
    • Microsoft Learn

Deploy infrastructure

This action will create resource groups, vnets, peering between vnets and AKS clusters to run all required workloads

make deploy-infra

Setup clusters

Install required Ingress controller in both clusters in order to deploy K8GB and demo application

make setup-clusters

Configure GSS-TSIG authentication for DNS updates

Before deploying K8GB and the demo workload, ensure required configurations on Windows DNS

Domain Controller config

• Ensure that the Network Security is configured only for AES256

• Ensure that the DNS Zone has only Secure updates option enabled

• Ensure that the DNS Zone has the option "Allow zone transfers" check with the option "To any server" under the tab Zone Transfers on the zone properties

• Create a new Active Directory user
  • The user should be created with "Encryptions options" for Kerberos AES256 encryption
  • The user needs to be added to the DNSAdmin group, or,
  • Select the zone that will have dynamic updates in DNS Manager, right click and select Properties. Under the Security tab, add the created user and add the permissions Write, Create all child objects and Delete all child objects

K8GB / ExternalDNS configuration

• ExternalDNS configuration
  • For communication with WindowsDNS, ExternalDNS should be configured with the RFC2136 provider with GSS-TSIG option
  • External DNS - RFC2126
  • A sample values.yaml for K8GB configuration can be found here.
    • Ensure that the following properties are updated with your values:
      • dnsZone
      • edgeDNSZone
      • edgeDNSServers
      • host - always use FQDN with GSS-TSIG, not IP address
      • kerberos-username
      • kerberos-password
      • kerberos-realm
  • At this moment ExternalDNS doesn't provide a way to use secrets as the source for the kerberos-password setting, so if you store the manifest in a git repo, please ensure that only required persons can access it

rfc2136:
  enabled: true
  rfc2136Opts:
    - host: AD-DC.mbcpk8gb.local #when using gssTsig, use the FQDN of the host, not an IP
    - port: 53
  rfc2136auth:
    insecure: 
      enabled: false
    tsig:
      enabled: false
      tsigCreds:
        - tsig-secret-alg: hmac-sha256
        - tsig-keyname: externaldns-key
    gssTsig:
      enabled: true
      gssTsigCreds:
        - kerberos-username: ad-user-account
        - kerberos-password: ad-user-account-password
        - kerberos-realm: mbcpcloud.lab

Install K8gb

This action will install K8gb in both clusters using the provided sample values.yaml for each cluster. Please ensure that the are correctly updated before execution

make deploy-k8gb

Install demo app

Deploys the sample Podinfo workload with failover GLSB configured using annotations in the Ingress resource samples. Ensure that the hosts on the samples are correctly updated before execution

make deploy-demo

Destroy lab

• Destroys the lab environment created for this sample

make destroy-infra