Filter out elements from a json object in in Terraform
Posted on October 19, 2022 by Adrian Wyssmann ‐ 4 min read
We would like to manage our own Azure Policy Initiatives with Terraform. Our own initatives shall be based on existing ones, but without deprecated policies.
What are Azure Policy Initiatives
Azure Policy allows you audit your infrastructure on Azure and enforce compliance.
Through its compliance dashboard, it provides an aggregated view to evaluate the overall state of the environment, with the ability to drill down to the per-resource, per-policy granularity. It also helps to bring your resources to compliance through bulk remediation for existing resources and automatic remediation for new resources.
Policies are grouped in initiatives:
An initiative definition is a collection of policy definitions that are tailored toward achieving a singular overarching goal.
There are several builtin initiatives which are constantly updated. But this also means, some of the policies part of the initiative are deprecated.
How to manage Policy Initiatives with Terraform
azurerm_policy_set_definition form the azurerm provider is the one resource, that manages policy sets or also know as policy initiatives. While the field parameters is used to provide the policy definition as a JSON object. AS mentioned in the introduction, we want to have our own initiatives - based on existing ones - but without deprecated policies.
Let’s say we want to create an initiative based AzureSecurityCenter initiative, we can use the [azure_policy_set_definition datasource] to access an existing policy
data "azurerm_policy_set_definition" "azuresecuirtybenchmark" {
display_name = "Azure Security Benchmark"
}
We can then access the parameter as follows
data.azurerm_policy_set_definition.azuresecuirtybenchmark.parameters
How to remove deprecated policies using Terraform
So far so good, so let’s see what we get:
locals {
decoded_policy_set_definition_params = jsondecode(data.azurerm_policy_set_definition.azuresecuirtybenchmark.parameters)
}
output "demo" {
value = local.decoded_policy_set_definition_params
}
This will show you the content:
Changes to Outputs:
+ demo = {
+ ASCDependencyAgentAuditLinuxEffect = {
+ allowedValues = [
+ "AuditIfNotExists",
+ "Disabled",
]
+ defaultValue = "AuditIfNotExists"
+ metadata = {
+ description = "Enable or disable Dependency Agent for Linux VMs"
+ displayName = "Audit Dependency Agent for Linux VMs monitoring"
}
+ type = "String"
...
From there let’s iterate over the root elements, by adding this
filtered_policy_set_definition_params = [
for v in decoded_policy_set_definition_params : v.type
]
However terraform plan results in this error
Error: Invalid reference
on local.tf line 18, in locals:
18: for v in decoded_policy_set_definition_params : s.type
A reference to a resource type must be followed by at least one attribute
This is cause the decoded_policy_set_definition_params is local, hence
filtered_policy_set_definition_params = [
for v in local.decoded_policy_set_definition_params : v
]
Works and if we adjust the output as follows
output "demo" {
value = local.filtered_policy_set_definition_params
}
We still get the same output:
Changes to Outputs:
+ demo = [
+ {
+ allowedValues = [
+ "AuditIfNotExists",
+ "Disabled",
]
+ defaultValue = "AuditIfNotExists"
+ metadata = {
+ description = "Enable or disable Dependency Agent for Linux VMs"
+ displayName = "Audit Dependency Agent for Linux VMs monitoring"
}
+ type = "String"
},
...
Now we want to filter out all elements which contain
metadata = {
deprecated = true
Following this post the recommended way is to use [lookup], which
retrieves the value of a single element from a map, given its key. If the given key does not exist, the given default value is returned instead.
As we want all that don’t have the attribute metadata.deprecated, we would use this
for v in local.decoded_policy_set_definition_params : s if lookup(v.metadata, "deprecated", null) == null
The result contains all elements, but we are missing the key of the element (the policy name)
Changes to Outputs:
+ demo = [
+ {
+ allowedValues = [
+ "AuditIfNotExists",
+ "Disabled",
]
+ defaultValue = "AuditIfNotExists"
+ metadata = {
+ description = "Enable or disable Dependency Agent for Linux VMs"
+ displayName = "Audit Dependency Agent for Linux VMs monitoring"
}
...
So when iterating over the json object we have to read carefully what it tells [here][result types]
The type of brackets around the for expression decide what type of result it produces.
[]produces a tuple{}result is an object and you must provide two result expressions that are separated by the=>symbol
So the adjusted expression is
filtered_policy_set_definition_params = {
for k, v in local.decoded_policy_set_definition_params : k=>v if lookup(v.metadata, "deprecated", null) == null
}
Which results in this
+ demo = {
+ ASCDependencyAgentAuditLinuxEffect = {
+ allowedValues = [
+ "AuditIfNotExists",
+ "Disabled",
]
+ defaultValue = "AuditIfNotExists"
+ metadata = {
+ description = "Enable or disable Dependency Agent for Linux VMs"
+ displayName = "Audit Dependency Agent for Linux VMs monitoring"
}
+ type = "String"
...
Now we can use this in our azurerm_policy_set_definition as an jsonencode-ed string:
resource "azurerm_policy_set_definition" "Example" {
name = "Example-Policy-Set"
policy_type = "Custom"
display_name = "Example-Policy-Set"
management_group_id = data.azurerm_management_group.Sandboxes.id
parameters = jsonencode(local.filtered_policy_set_definition_params)
dynamic "policy_definition_reference" {
for_each = data.azurerm_policy_set_definition.test.policy_definition_reference
content {
parameter_values = policy_definition_reference.value.parameter_values
policy_definition_id = policy_definition_reference.value.policy_definition_id
reference_id = policy_definition_reference.value.reference_id
}
}
}