Inject Secrets into Terraform Environment using 1Password

Following a recent upgrade within my homelab to Proxmox VE v8.0 I had a need to refactor my Terraform configurations.¹

This was necessary because my previous Proxmox provider for Terraform, Telmate/proxmox, has not been kept up to date and would fail to work properly with Proxmox VE’s API following the v8 upgrade. Luckily, a second provider bpg/proxmox exists and is seeing regular updates so I can move over to that.

This post is not about that migration, but instead about a neat little trick I picked up during it. Credit to the various blogs and Mastodon posts that got me to this solution, which I assure you I did not discover all on my own, but nevertheless want to document here for posterity and to help others searching for this solution.

One of the typical files² in a Terraform configuration is version.tf, which describes the version of Terraform and related providers being used.

terraform {
  required_providers {
    proxmox = {
      source  = "bpg/proxmox"
      version = "0.39.0"
    }
  }
  required_version = ">= 0.13"
}

provider "proxmox" {
  endpoint = "https://proxmox.lan:8006/"
  api_token = "${var.PROXMOX_TOKEN_ID}=${var.PROXMOX_TOKEN_SECRET}"
  insecure = false
  tmp_dir  = "/tmp"
}

The block starting provider "proxmox" requires an API Token, which I reference as a combination of two variables: "${var.PROXMOX_TOKEN_ID}=${var.PROXMOX_TOKEN_SECRET}".

Those variables are defined within another file in that same folder, variables.tf:

variable "PROXMOX_TOKEN_ID" {
  description = "Proxmox API Token ID, read from the .env_vars"
  type = string
  sensitive = true
}

variable "PROXMOX_TOKEN_SECRET" {
  description = "Proxmox API Token secret, read from the .env_vars"
  type = string
  sensitive = true
}

Which describes, but does not define those variables. Note that I could define the values here, but this file is something that gets checked into source control (Git) and could be replicated to a remote repository in the “cloud” (aka, someone else’s computer). I don’t want my secrets in there!

I can get around including them in the variables.tf file by instead putting them into the environment. This is accomplished in two parts, first by creating a file .env_vars:³

TF_VAR_PROXMOX_TOKEN_ID="op://SomeVault/Proxmox Credential/Auth Token/Token ID"
TF_VAR_PROXMOX_TOKEN_SECRET="op://SomeVault/Proxmox Credential/Auth Token/secret"

This file has some magic ✨ in it.

• First, I am using a special syntax for the environment variable itself – TF_VAR_... – which tells Terraform to look for anything following the TF_VAR_ value in it’s environment, which in my example is PROXMOX_TOKEN_ID and PROXMOX_TOKEN_SECRET.
• Second, I’m using the 1Password syntax for referring to secret values stored in my 1Password Vaults.

The second part is to run the Terraform command with a little extra syntax:

op run --env-file=".env_vars" -- terraform plan

This also uses 1Password syntax to inject variables into the environment, from my 1Password Vault, according to the environment variables I have defined in the .env_vars file. Then, the terraform command can access those specific environment variables. 1Password makes these environment variables available only in the subprocess running the command, helping ensure your secrets stay secret and do not leak out into the environment.

This system could be scaled to support multiple environments (dev, staging, prod) as well! 1Password CLI can expand environment variables within the environment the op run command executes. This means you could define .env_vars like so:

TF_VAR_PROXMOX_TOKEN_ID="op://$APP_ENV/Proxmox Credential/Auth Token/Token ID"
TF_VAR_PROXMOX_TOKEN_SECRET="op://$APP_ENV/Proxmox Credential/Auth Token/secret"

And then execute terraform across different environments.

# Dev
APP_ENV=dev op run --env-file=".env_vars" -- terraform plan

# Staging
APP_ENV=staging op run --env-file=".env_vars" -- terraform plan

# Production
APP_ENV=prod op run --env-file=".env_vars" -- terraform plan

In this example, there are three Vaults defined – dev, staging, and prod. Each Vault contains an item named “Proxmox Credential” which has an “Auth Token” section and two fields defined, “Token ID” and “secret”.

That’s quite nifty!