Using an Azure DevOps Service Connection to Authenticate from a Terraform local-exec Provisioner

At my current job, we use Terraform to define Azure infrastructure as code and Azure DevOps pipelines to deploy it. The Azure Provider for Terraform is quite robust and can probably do 95% or more of whatever we need to configure in Azure, but what about the remaining 5%?

This is where Terraform's null_resource and provisioners come into play. Since we're managing Azure resources, PowerShell is an obvious choice for a scripting language. What's less obvious is the best way to authenticate PowerShell commands against Azure from within a Terraform configuration.

You could pass secrets into your Terraform configuration at runtime, but if you ever try passing a sensitive value into a provisioner script, you will notice that terraform suppresses all of your script's output with a message that looks something like this (local-exec): (output suppressed due to sensitive value in config). You could mark the values as nonsensitive to make the output visible, but doing so risks exposing sensitive values in your logs. It also raises the question of how to pass the secrets into the configuration in the first place; because many of us use tfvars files to pass in configuration variables, it might become tempting to store secrets in code (just don't).

This got me wondering if it was possible to use an Azure DevOps service connection (like the one Terraform already uses) to authenticate against Azure from within our provisioner script. Fortunately, it is.

The Az Module and Azure DevOps Pipelines

The Azure PowerShell task

If you've used PowerShell to manage Azure resources from an Azure DevOps pipeline before, then you're probably familiar with the Azure PowerShell task. Notice that the task allows you to select a service connection:

Behind the scenes, this task uses Connect-AzAccount during its initialization with parameters from the service connection to create a context used to authenticate any subsequent commands from the Az module.

A note about Azure PowerShell contexts

By default, Azure PowerShell contexts are saved and can be reused in subsequent PowerShell sessions by the same user without re-running Connect-AzAccount for a period of time. We can leverage this behavior to authenticate the PowerShell provisioner script in our Terraform config.

The Az Module

The Azure DevOps hosted agents have several versions of the Az module installed out of the box, these are what the Azure PowerShell task uses when it runs, but these preinstalled versions are outside the standard PSModulePath. While we could figure out the location of these modules, I would consider that unreliable and subject to change without notice. Instead we should install it ourselves from the PowerShell gallery.

The Az module is actually a collection of modules, and any given script is almost certainly not going to use all of them. To make your pipelines run more quickly, you can figure out which modules you're actually using and install them individually. At a minimum, you will need the Az.Accounts module.

Putting it all together (The TL;DR)

Now let's take what we know and use it to build a pipeline.

Create an Az Context

Add an Azure PowerShell script step to your pipeline, name it something like "Create Az Context" for clarity, select your service connection, select "Inline Script". You can add a write-host message if you want.

As I mentioned earlier, this task creates an Azure context during its initialization, and that context can be reused in subsequent PowerShell sessions.

Install the Az Module

Add a PowerShell Script task to your pipeline, name it something like "Install Az Modules", add your Install-Module commands.

You can install everything...

Install-Module Az -Scope CurrentUser -Force

Or just what you need...

# You will always need the Az.Accounts module
Install-Module Az.Accounts -Scope CurrentUser -Force

# As an example, you might need to use commands from the Az.Sql module 
# in your provisioner script
Install-Module Az.Sql -Scope CurrentUser -Force

Install Terraform

Add a Terraform tool installer task to your pipeline, you may specify latest or a specific version.

Apply The Terraform Config

Assuming that the build artifact that your release pipeline is consuming contains a plan file and the .terraform directory generated by your build job, then all you need to do now is add a task to apply your Terraform config.

Any Az PowerShell commands used in your Terraform configuration's provisioner scripts will authenticate automatically using the context that was created in the "Create Az Context" step earlier in the pipeline, and will use the modules you installed in the "Install Az Modules" step.