Skip to content
/ terraform-azurerm-terraform-enterprise-aks-hvd Public
generated from hashicorp/hvd-module-template

A Terraform module for provisioning and installing Terraform Enterprise on Azure AKS as described in HashiCorp Validated Designs

License

MPL-2.0 license
1 star 7 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

hashicorp/terraform-azurerm-terraform-enterprise-aks-hvd

BranchesTags

Repository files navigation

Terraform Enterprise HVD on Azure AKS

Terraform module aligned with HashiCorp Validated Designs (HVD) to deploy Terraform Enterprise on Azure Kubernetes Service (AKS). This module supports bringing your own AKS cluster, or optionally creating a new AKS cluster dedicated to running TFE. This module does not use the Kubernetes or Helm Terraform providers, but rather includes Post Steps for the application layer portion of the deployment leveraging the kubectl and helm CLIs.

Prerequisites

General

  • TFE license file (e.g. terraform.hclic)
  • Terraform CLI (version >= 1.9) installed on clients/workstations that will be used to deploy TFE
  • General understanding of how to use Terraform (Community Edition)
  • General understanding of how to use Azure cloud
  • General understaning of how to use Kubernetes and Helm
  • az CLI installed on workstation
  • kubectl CLI and helm CLI installed on workstation
  • git CLI and Visual Studio Code code editor are strongly recommended
  • Azure subscription that TFE will be deployed in
  • Azure blob storage account for AzureRM remote state backend that will be used to manage the Terraform state of this TFE deployment (out-of-band from the TFE application) via Terraform CLI (Community Edition)

Networking

  • Azure virtual network (VNet) ID that TFE will be deployed in
  • TFE load balancer subnet ID (this can be the same as the AKS subnet ID if you prefer)
  • Static IP address for TFE load balancer (load balancer is created/managed by Helm/Kubernetes)
  • AKS subnet ID for AKS cluster where TFE pods will be running. The AKS/TFE pods subnet should be configured as follows:
    • Service endpoints configured for Microsoft.Sql and Microsoft.Storage in the event that you disable the creation of private endpoints by this module (default behavior is to create private endpoints, but having the service endpoints enabled as a fallback option is a good practice and will not negatively impact anything)
  • If your AKS cluster is private, then your clients/workstations must be able to access the AKS cluster control plane via kubectl and helm CLIs
  • Database subnet ID for PostgreSQL flexible server. The database subnet should be configured as follows:
    • Allow the creation of private endpoints (private_endpoint_network_policies_enabled = false)
    • Service delegation configured for PostgreSQL flexible servers (Microsoft.DBforPostgreSQL/flexibleServers) for join action (Microsoft.Network/virtualNetworks/subnets/join/action)
    • Service endpoint configured for Microsoft.Storage
  • Redis subnet ID for Azure cache for Redis. The Redis subnet should be configured to allow the creation of private endpoints (private_endpoint_network_policies_enabled = false)

Network security group (NSG)/firewall rules

  • Allow TCP/443 ingress to TFE load balancer subnet from CIDR ranges of TFE users/clients, VCS, and other systems that needs to reach TFE
  • (Optional) Allow TCP/9091 (HTTPS) and/or TCP/9090 (HTTP) ingress to AKS/TFE pods subnet from CIDR ranges of your monitoring/observability tool (for scraping TFE metrics endpoints)
  • Allow TCP/8443 (HTTPS) and TCP/8080 (HTTP) ingress to AKS/TFE pods subnet from TFE load balancer subnet (for TFE application traffic)
  • Allow TCP/5432 ingress to database subnet from AKS/TFE pods subnet (for PostgreSQL traffic)
  • Allow TCP/6380 ingress to Redis cache subnet from AKS/TFE pods subnet (for Redis TLS traffic)
  • Allow TCP/8201 between nodes on AKS/TFE pods subnet (for TFE embedded Vault internal cluster traffic)
  • Allow TCP/443 egress to Terraform endpoints listed here from AKS/TFE pods subnet

TLS certificates

  • TLS certificate (e.g. cert.pem) and private key (e.g. privkey.pem) that matches your chosen fully qualified domain name (FQDN) for TFE
    • TLS certificate and private key must be in PEM format
    • Private key must not be password protected
  • TLS certificate authority (CA) bundle (e.g. ca_bundle.pem) corresponding with the CA that issues your TFE TLS certificates
    • CA bundle must be in PEM format
    • You may include additional certificate chains corresponding to external systems that TFE will make outbound connections to (e.g. your self-hosted VCS, if its certificate was issued by a different CA than your TFE certificate)

đź“ť Note: The TLS certificate and private key will be created as Kubernetes secrets during the Post Steps.

Key Vault secret

The following bootstrap secret needs to exist in an Azure Key Vault:

  • TFE database password (to be applied to PostgreSQL flexible server) - both the Key Vault ID and secret name are required

Compute (optional)

If you plan to create a new AKS cluster using this module, then you may skip this section. Otherwise:

  • AKS cluster
    • Network Contributor role scoped to TFE pods subnet and (if applicable) TFE load balancer subnet

Usage

  1. Create/configure/validate the applicable prerequisites.

  2. Nested within the examples directory are subdirectories containing ready-made Terraform configurations for example scenarios on how to call and deploy this module. To get started, choose the example scenario that most closely matches your requirements. You can customize your deployment later by adding additional module inputs as you see fit (see the Deployment-Customizations doc for more details).

  3. Copy all of the Terraform files from your example scenario of choice into a new destination directory to create your Terraform configuration that will manage your TFE deployment. This is a common directory structure for managing multiple TFE deployments:

    .
└── environments
    ├── production
    │   ├── backend.tf
    │   ├── main.tf
    │   ├── outputs.tf
    │   ├── terraform.tfvars
    │   └── variables.tf
    └── sandbox
        ├── backend.tf
        ├── main.tf
        ├── outputs.tf
        ├── terraform.tfvars
        └── variables.tf

    đź“ť Note: In this example, the user will have two separate TFE deployments; one for their sandbox environment, and one for their production environment. This is recommended, but not required.

  4. (Optional) Uncomment and update the AzureRM remote state backend configuration provided in the backend.tf file with your own custom values. While this step is highly recommended, it is technically not required to use a remote backend config for your TFE deployment (if you are in a sandbox environment, for example).

  5. Populate your own custom values into the terraform.tfvars.example file that was provided (in particular, values enclosed in the <> characters). Then, remove the .example file extension such that the file is now named terraform.tfvars.

  6. Navigate to the directory of your newly created Terraform configuration for your TFE deployment, and run terraform init, terraform plan, and terraform apply.

The TFE infrastructure resources have now been created. Next comes the application layer portion of the deployment (which we refer to as the Post Steps), which will involve interacting with your AKS cluster via kubectl and installing the TFE application via helm.

Post Steps

  1. Authenticate to your AKS cluster:

    az login
az account set --subscription <Subscription Name or ID>
az aks get-credentials --resource-group <Resource Group> --name <AKS Cluster Name>

  2. Create the Kubernetes namespace for TFE:

    kubectl create namespace tfe

    đź“ť Note: You can name it something different than tfe if you prefer. If you do name it differently, be sure to update your value of the tfe_kube_namespace input variable accordingly.

  3. Create the required secrets for your TFE deployment within your new Kubernetes namespace for TFE. There are several ways to do this, whether it be from the CLI via kubectl, or another method involving a third-party secrets helper/tool. See the kubernetes-secrets docs for details on the required secrets and how to create them.

  4. This Terraform module will automatically generate a Helm overrides file within your Terraform working directory named ./helm/module_generated_helm_overrides.yaml. This Helm overrides file contains values interpolated from some of the infrastructure resources that were created by Terraform in step 6. Within the Helm overrides file, update or validate the values for the remaining settings that are enclosed in the <> characters. You may also add any additional configuration settings into your Helm overrides file at this time (see the helm-overrides doc for more details).

  5. Now that you have customized your module_generated_helm_overrides.yaml file, rename it to something more applicable to your deployment, such as prod_tfe_overrides.yaml (or whatever you prefer). Then, within your terraform.tfvars file, set the value of create_helm_overrides_file to false, as we no longer want the Terraform module to manage this file or generate a new one on a subsequent Terraform run.

  6. Add the HashiCorp Helm registry:

    helm repo add hashicorp https://helm.releases.hashicorp.com

đź“ť Note: If you have already added the hashicorp Helm repository, you should run helm repo update hashicorp to ensure that you have the latest version.

  1. Install the TFE application via helm:

    helm install terraform-enterprise hashicorp/terraform-enterprise --namespace <TFE_NAMESPACE> --values <TFE_OVERRIDES_FILE>

  2. Verify the TFE pod(s) are successfully starting:

    View the events within the namespace:

    kubectl get events --namespace <TFE_NAMESPACE>

    View the pod(s) within the namespace:

    kubectl get pods --namespace <TFE_NAMESPACE>

    View the logs from the pod:

    kubectl logs <TFE_POD_NAME> --namespace <TFE_NAMESPACE> -f

  3. Create a DNS record for your TFE FQDN. The DNS record should resolve to your TFE load balancer, depending on how the load balancer was configured during your TFE deployment:

    • If you are using a Kubernetes service of type LoadBalancer (what the module-generated Helm overrides defaults to), the DNS record should resolve to the static IP address of your TFE load balancer:

      kubectl get services --namespace <TFE_NAMESPACE>

    • If you are using a custom Kubernetes ingress (meaning you customized your Helm overrides in step 10), the DNS record should resolve to the IP address of your ingress controller load balancer.

      kubectl get ingress <INGRESS_NAME> --namespace <INGRESS_NAMESPACE>

  4. Verify the TFE application is ready:

    curl https://<TFE_FQDN>/_health_check

  5. Follow the remaining steps here to finish the installation setup, which involves creating the initial admin user.

Docs

Below are links to various docs related to the customization and management of your TFE deployment:

Module support

This open source software is maintained by the HashiCorp Technical Field Organization, independently of our enterprise products. While our Support Engineering team provides dedicated support for our enterprise offerings, this open source software is not included.

  • For help using this open source software, please engage your account team.
  • To report bugs/issues with this open source software, please open them directly against this code repository using the GitHub issues feature.

Please note that there is no official Service Level Agreement (SLA) for support of this software as a HashiCorp customer. This software falls under the definition of Community Software/Versions in your Agreement. We appreciate your understanding and collaboration in improving our open source projects.

Requirements

Name Version
terraform >= 1.9
azurerm ~> 3.116
random ~> 3.6

Providers

Name Version
azurerm ~> 3.116
local n/a

Resources

Name Type
azurerm_dns_a_record.tfe resource
azurerm_federated_identity_credential.tfe_kube_service_account resource
azurerm_kubernetes_cluster.tfe resource
azurerm_kubernetes_cluster_node_pool.tfe resource
azurerm_postgresql_flexible_server.tfe resource
azurerm_postgresql_flexible_server_configuration.tfe resource
azurerm_postgresql_flexible_server_database.tfe resource
azurerm_private_dns_a_record.blob_storage resource
azurerm_private_dns_a_record.redis resource
azurerm_private_dns_a_record.tfe resource
azurerm_private_dns_zone.blob_storage resource
azurerm_private_dns_zone.postgres resource
azurerm_private_dns_zone.redis resource
azurerm_private_dns_zone_virtual_network_link.blob_storage resource
azurerm_private_dns_zone_virtual_network_link.postgres resource
azurerm_private_dns_zone_virtual_network_link.redis resource
azurerm_private_endpoint.blob_storage resource
azurerm_private_endpoint.redis resource
azurerm_redis_cache.tfe resource
azurerm_resource_group.tfe resource
azurerm_role_assignment.aks_network_contributor_aks_subnet resource
azurerm_role_assignment.aks_network_contributor_tfe_lb_subnet resource
azurerm_role_assignment.tfe_blob_storage resource
azurerm_storage_account.tfe resource
azurerm_storage_account_network_rules.tfe resource
azurerm_storage_container.tfe resource
azurerm_user_assigned_identity.tfe resource
local_file.helm_overrides_values resource
azurerm_dns_zone.tfe data source
azurerm_key_vault_secret.tfe_database_password data source
azurerm_private_dns_zone.tfe data source

Inputs

Name Description Type Default Required
aks_api_server_authorized_ip_ranges List of IP ranges that are allowed to access the AKS API server (control plane). list(string) [] no
aks_default_node_pool_max_surge The maximum number of nodes that can be added during an upgrade. string "10%" no
aks_default_node_pool_name Name of default node pool. string "default" no
aks_default_node_pool_node_count Number of nodes to run in the AKS default node pool. number 2 no
aks_default_node_pool_vm_size Size of the virtual machines within the AKS default node pool. string "Standard_D8ds_v5" no
aks_dns_service_ip The IP address assigned to the AKS internal DNS service. string "10.1.0.10" no
aks_kubernetes_version Kubernetes version for AKS cluster. string "1.29.6" no
aks_oidc_issuer_enabled Boolean to enable OIDC issuer for the AKS cluster. bool true no
aks_role_based_access_control_enabled Boolean to enable Role-Based Access Control (RBAC) for the AKS cluster. bool true no
aks_service_cidr IP range for Kubernetes services that can be used by AKS cluster. string "10.1.0.0/16" no
aks_subnet_id Subnet ID for AKS cluster. string null no
aks_tfe_node_pool_name Name of TFE node pool. Only valid when create_aks_tfe_node_pool is true. string "tfeaksnodes" no
aks_tfe_node_pool_node_count Number of nodes in the AKS TFE node pool. Only valid when create_aks_tfe_node_pool is true. number 2 no
aks_tfe_node_pool_vm_size Size of virtual machines in the AKS TFE node pool. Only valid when create_aks_tfe_node_pool is true. string "Standard_D8ds_v5" no
aks_workload_identity_enabled Boolean to enable Workload Identity for the AKS cluster. bool true no
availability_zones List of Azure availability zones to spread TFE resources across. set(string) 
[
  "1",
  "2",
  "3"
]
 no
common_tags Map of common tags for taggable Azure resources. map(string) {} no
create_aks_cluster Boolean to create a new AKS cluster for this TFE deployment. bool false no
create_aks_tfe_node_pool Boolean to create a new node pool for TFE in the AKS cluster. bool false no
create_blob_storage_private_endpoint Boolean to create a private endpoint and private DNS zone for TFE Storage Account. bool true no
create_helm_overrides_file Boolean to generate a YAML file from template with Helm overrides values for TFE deployment. bool true no
create_postgres_private_endpoint Boolean to create a private endpoint and private DNS zone for PostgreSQL Flexible Server. bool true no
create_redis_private_endpoint Boolean to create a private DNS zone and private endpoint for Redis cache. bool true no
create_resource_group Boolean to create a new resource group for this TFE deployment. bool true no
create_tfe_private_dns_record Boolean to create a DNS record for TFE in a private Azure DNS zone. A private_dns_zone_name must also be provided when true. bool false no
create_tfe_public_dns_record Boolean to create a DNS record for TFE in a public Azure DNS zone. A public_dns_zone_name must also be provided when true. bool false no
db_subnet_id Subnet ID for PostgreSQL flexible server database. string n/a yes
friendly_name_prefix Friendly name prefix used for uniquely naming all Azure resources for this deployment. Most commonly set to either an environment (e.g. 'sandbox', 'prod'), a team name, or a project name. string n/a yes
is_govcloud_region Boolean indicating if this TFE deployment is in an Azure Government Cloud region. bool false no
is_secondary_region Boolean indicating whether this TFE deployment is for 'primary' region or 'secondary' region. bool false no
location Azure region for this TFE deployment. string n/a yes
postgres_backup_retention_days Number of days to retain backups of PostgreSQL Flexible Server. number 35 no
postgres_create_mode Determines if the PostgreSQL Flexible Server is being created as a new server or as a replica. string "Default" no
postgres_enable_high_availability Boolean to enable ZoneRedundant high availability with PostgreSQL database. bool false no
postgres_geo_redundant_backup_enabled Boolean to enable PostreSQL geo-redundant backup configuration in paired Azure region. bool true no
postgres_maintenance_window Map of maintenance window settings for PostgreSQL flexible server. map(number) 
{
  "day_of_week": 0,
  "start_hour": 0,
  "start_minute": 0
}
 no
postgres_primary_availability_zone Number for the availability zone for the db to reside in number 1 no
postgres_secondary_availability_zone Number for the availability zone for the db to reside in for the secondary node number 2 no
postgres_sku PostgreSQL database SKU. string "GP_Standard_D4ds_v4" no
postgres_storage_mb Storage capacity of PostgreSQL Flexible Server (unit is megabytes). number 65536 no
postgres_version PostgreSQL database version. number 15 no
private_dns_zone_name Name of existing private Azure DNS zone to create DNS record in. Required when create_tfe_private_dns_record is true. string null no
private_dns_zone_rg_name Name of Resource Group where private_dns_zone_name resides. Required when create_tfe_private_dns_record is true. string null no
public_dns_zone_name Name of existing public Azure DNS zone to create DNS record in. Required when create_tfe_public_dns_record is true. string null no
public_dns_zone_rg_name Name of Resource Group where public_dns_zone_name resides. Required when public_dns_zone_name is not null. string null no
redis_capacity The size of the Redis cache to deploy. Valid values for a SKU family of C (Basic/Standard) are 0, 1, 2, 3, 4, 5, 6, and for P (Premium) family are 1, 2, 3, 4. number 1 no
redis_enable_authentication Boolean to enable authentication to the Redis cache. bool true no
redis_enable_non_ssl_port Boolean to enable port non-SSL port 6379 for Redis cache. bool false no
redis_family The SKU family/pricing group to use. Valid values are C (for Basic/Standard SKU family) and P (for Premium). string "P" no
redis_min_tls_version Minimum TLS version to use with Redis cache. string "1.2" no
redis_sku_name Which SKU of Redis to use. Options are 'Basic', 'Standard', or 'Premium'. string "Premium" no
redis_subnet_id Subnet ID for Azure cache for Redis. string n/a yes
redis_version Redis cache version. Only the major version is needed. number 6 no
resource_group_name Name of resource group for this TFE deployment. Must be an existing resource group if create_resource_group is false. string n/a yes
secondary_aks_subnet_id AKS subnet ID of existing TFE AKS cluster in secondary region. Used to allow AKS TFE nodes in secondary region access to TFE storage account in primary region. string null no
storage_account_ip_allow List of CIDRs allowed to access TFE Storage Account. list(string) [] no
storage_account_public_network_access_enabled Boolean to enable public network access to Azure Blob Storage Account. Needs to be true for initial creation. Set to false after initial creation. bool true no
storage_account_replication_type Which type of replication to use for TFE Storage Account. string "ZRS" no
tfe_database_name PostgreSQL database name for TFE. string "tfe" no
tfe_database_parameters Additional parameters to pass into the TFE database settings for the PostgreSQL connection URI. string "sslmode=require" no
tfe_database_password_keyvault_id Resource ID of the Key Vault that contains the TFE database password. string n/a yes
tfe_database_password_keyvault_secret_name Name of the secret in the Key Vault that contains the TFE database password. string n/a yes
tfe_database_user Name of PostgreSQL TFE database user to create. string "tfe" no
tfe_dns_record_target Target of the TFE DNS record. This should be the IP address that the TFE FQDN resolves to. string null no
tfe_fqdn Fully qualified domain name of TFE instance. This name should eventually resolve to the TFE load balancer DNS name or IP address and will be what clients use to access TFE. string n/a yes
tfe_http_port HTTP port number that the TFE application will listen on within the TFE pods. It is recommended to leave this as the default value. number 8080 no
tfe_https_port HTTPS port number that the TFE application will listen on within the TFE pods. It is recommended to leave this as the default value. number 8443 no
tfe_kube_namespace Kubernetes namespace for TFE deployment. string "tfe" no
tfe_kube_service_account Kubernetes service account for TFE deployment. string "tfe" no
tfe_lb_subnet_id Subnet ID for TFE load balancer. This can be the same as the AKS subnet ID if desired. The TFE load balancer is created/managed by Helm/Kubernetes. string null no
tfe_metrics_http_port HTTP port number that the TFE metrics endpoint will listen on within the TFE pods. It is recommended to leave this as the default value. number 9090 no
tfe_metrics_https_port HTTPS port number that the TFE metrics endpoint will listen on within the TFE pods. It is recommended to leave this as the default value. number 9091 no
tfe_object_storage_azure_use_msi Boolean to use TFE user-assigned managed identity (MSI) to access TFE blob storage. If true, aks_workload_identity_enabled must also be true. bool false no
tfe_primary_resource_group_name Name of existing resource group of TFE deployment in primary region. Only set when is_secondary_region is true. string null no
tfe_primary_storage_account_name Name of existing TFE storage account in primary region. Only set when is_secondary_region is true. string null no
tfe_primary_storage_container_name Name of existing TFE storage container (within TFE storage account) in primary region. Only set when is_secondary_region is true. string null no
vnet_id VNet ID where TFE resources will reside. string n/a yes

Outputs

Name Description
aks_cluster_name Name of the AKS cluster.
resource_group_name Name of the resource group.
tfe_database_host Fully qualified domain name (FQDN) and port of the PostgreSQL flexible server.
tfe_database_name Name of the PostgreSQL flexible server TFE database.
tfe_database_password Password of the PostgreSQL flexible server TFE database.
tfe_database_password_base64 Base64-encoded password of the PostgreSQL flexible server TFE database.
tfe_database_user Username of the PostgreSQL flexible server TFE database.
tfe_object_storage_azure_account_key Primary access key of the storage account for TFE object storage.
tfe_object_storage_azure_account_key_base64 Base64-encoded primary access key of the storage account for TFE object storage.
tfe_object_storage_azure_account_name Name of the storage account for TFE object storage.
tfe_object_storage_azure_client_id Client ID of the managed identity (MSI) used by TFE to access the storage account.
tfe_object_storage_azure_container Name of the storage account container for TFE object storage.
tfe_object_storage_azure_use_msi Boolean indicating whether TFE is using a managed identity (MSI) to access the storage account.
tfe_private_dns_record_fqdn Private DNS record for TFE.
tfe_public_dns_record_fqdn Public DNS record for TFE.
tfe_redis_host Hostname of the Redis cache.
tfe_redis_password Primary access key of the Redis cache.
tfe_redis_password_base64 Base64-encoded primary access key of the Redis cache.
tfe_redis_use_auth Boolean indicating whether TFE is using authentication to access the Redis cache.

About

A Terraform module for provisioning and installing Terraform Enterprise on Azure AKS as described in HashiCorp Validated Designs

Resources

Readme

License

MPL-2.0 license

Code of conduct

Code of conduct

Security policy

Security policy
Activity
Custom properties

Stars

1 star

Watchers

2 watching

Forks

7 forks
Report repository

Releases 2

0.2.0 Latest
Jun 27, 2025
+ 1 release

Packages

No packages published

Contributors 5

Languages