IaC Drift Detection: Terraform, OpenTofu, and the Config-Crawl Problem

November 9, 2026 · By Yuval Glasman

Sources

Official Documentation

In a mature platform organization, the Git repository is the singular source of truth. But in production, that truth is often a lie. Infrastructure Drift is the divergence between the desired state defined in your IaC (Terraform/OpenTofu) and the actual state of your cloud provider.

For a Security Architect, drift isn't just a "sync issue"—it's an unapproved change. In a regulated environment (SOC2, PCI-DSS), drift is a compliance failure. If a developer opens a security group port manually during a 3 AM incident and forgets to codify it, you have a permanent hole in your perimeter that your security scanners might miss, but your IaC definitely does.

The Mechanics: How Drift is Actually Detected

Terraform and OpenTofu do not "watch" your infrastructure in real-time. They rely on a pull-based refresh model. When you run a plan, the core engine calls the Read function of every provider resource in the state file.


# The industry-standard primitive for CI-based drift detection
terraform plan -refresh-only -detailed-exitcode

The -detailed-exitcode flag is the only way to automate this without fragile regex parsing of stdout:

0: No changes. The world matches the code.
1: Error. The plan failed (API timeout, credential expiry).
2: Drift detected. The world has diverged from the code.

Observe vs. Act: The Critical Distinction

A common junior mistake is scheduling a terraform apply -auto-approve to \"fix drift.\" This is dangerous. If a resource was deleted out-of-band (e.g., a production DB instance), a blind apply will attempt to recreate it, potentially causing catastrophic data loss or downtime. Detection must be separated from remediation.

The Config-Crawl Problem

Config-Crawl is the slow, entropic decay of your IaC. It starts with a \"temporary\" manual change to fix a production outage, followed by a \"quick\" tag update via the console. Over six months, your state file becomes a ghost of the actual infrastructure.

This creates two primary security risks:

Shadow Infrastructure: Resources created manually that aren't in state. Since the core only reads what's in the state file, these resources are invisible to drift detection.
The False Sense of Security: Your security auditors look at your Git repo and see a hardened perimeter, while the actual cloud environment is wide open.

The Drift Cycle Architecture

\n+----------------+       +------------------+       +------------------+ \n|   Git Repo     | ----> |  Provisioning     | ----> |  Cloud Provider  | \n| (Desired State)|       | (Terraform/Tofu)  |       |  (Actual State)   | \n+----------------+       +------------------+       +------------------+ \n                                    ^                        | \n                                    |                        | \n                                    |       [ DRIFT ] <------+ \n                                    |   (Manual Change) \n                                    | \n                        +-----------+-----------+ \n                        |   Drift Pipeline       | \n                        | 1. Observe (Plan)      | \n                        | 2. Classify (JSON)     | \n                        | 3. Remediate (Target)  | \n                        +------------------------+ \n

Classification Framework

Not all drift is created equal. Treating a tag change the same as an IAM policy change leads to \"alert fatigue\" and operational toil.

\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n

Bucket	Examples	Risk Level	Remediation Strategy
Cosmetic	Tags, Descriptions, Resource Names	Low	Auto-remediate via targeted apply
Security-Relevant	SG Rules, IAM Policies, Public Access Blocks	Critical	Immediate Page → Human Review
Structural	Resource Deleted, Instance Type changed	High	Human Review (check for data loss)
Provider Noise	Computed Timestamps, Provider IDs	None	Suppress via `ignore_changes`

Implementation Guide

1. The Scheduled Drift Detector (GitHub Actions)


name: Infrastructure Drift Detection
on:
  schedule:
    - cron: '0 0 * * *' # Daily at midnight
  workflow_dispatch:

jobs:
  detect-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      
      - name: Terraform Init
        run: terraform init

      - name: Check for Drift
        id: drift
        run: |
          terraform plan -refresh-only -detailed-exitcode -out=drift.tfplan
          echo "exit_code=$?" >> $GITHUB_OUTPUT

      - name: Handle Drift
        if: steps.drift.outputs.exit_code == '2'
        run: |
          terraform show -json drift.tfplan > drift.json
          # Trigger alert (e.g., Slack or PagerDuty)
          curl -X POST -H 'Content-type: application/json' \\
          --data '{\"text\":\"🚨 Infrastructure Drift Detected in Production!\"}' \\
          $

2. Filtering for Security-Critical Drift

To avoid alert fatigue, use a script to parse the JSON plan and only page for security resources.


#!/bin/bash
# Parse tfplan.json and alert only on critical resource types
PLAN_JSON=\"drift.json\"

# Critical resources that should NEVER drift
CRITICAL_TYPES=(\"aws_security_group_rule\" \"aws_iam_policy\" \"aws_iam_role\" \"aws_s3_bucket_public_access_block\")

# Extract changed resources
CHANGES=$(jq -r '.resource_changes[] | select(.change.actions[] == \"update\") | .type' $PLAN_JSON)

for TYPE in $CHANGES; do
  for CRITICAL in \"${CRITICAL_TYPES[@]}\"; do
    if [[ \"$TYPE\" == \"$CRITICAL\" ]]; then
      echo \"CRITICAL DRIFT: $TYPE has been modified out-of-band!\"
      # Exit with error to fail the CI build and trigger alerts
      exit 1
    fi
  done
done

echo \"No critical drift detected.\"
exit 0

3. Enforcing the Baseline with OPA

Prevent drift from being codified by using Open Policy Agent (OPA) to ensure that any \"reconciliation\" doesn't accidentally open the perimeter.


package terraform.drift

import future.keywords.if

# Deny any change that opens a security group to the world
deny[msg] {
    resource := input.resource_changes[_]
    resource.type == \"aws_security_group_rule\"
    resource.change.after.cidr_blocks[_] == \"0.0.0.0/0\"
    msg := sprintf(\"Drift reconciliation attempt violates security baseline: %s opens to 0.0.0.0/0\", [resource.address])
}

Production Realities

The State Deletion Trap: If a resource is deleted out-of-band and the provider's read returns not-found, the refresh removes it from state. An apply will then try to recreate it. If this is a stateful resource (Database, KMS key), you have just created a data-loss event. Always review structural drift manually.
Provider Noise: Some providers update metadata (like updated_at) on every read. Use lifecycle { ignore_changes = [...] } aggressively for these fields to avoid 100% false-positive drift rates.
Credential Bottlenecks: Running plan -refresh-only across 500 workspaces every hour will hit Cloud API rate limits. Implement jitter and staggered schedules.
The \"Ghost\" Resource: Drift detection ONLY works for resources in state. If someone creates a new S3 bucket manually, Terraform will never tell you. You need a secondary "Shadow Resource" scanner (e.g., CloudCustodian) to find unmanaged resources.

Who Should Skip This

Immutable Infrastructure Purists: If you destroy and recreate your entire environment on every deploy (Blue/Green), drift is a non-issue because the environment's lifespan is shorter than your scan interval.
Trivial Environments: If you have < 20 resources and one operator, a manual terraform plan once a week is sufficient.

The Mechanics: How Drift is Actually Detected

Observe vs. Act: The Critical Distinction

The Config-Crawl Problem

The Drift Cycle Architecture

Classification Framework

Implementation Guide

1. The Scheduled Drift Detector (GitHub Actions)

2. Filtering for Security-Critical Drift

3. Enforcing the Baseline with OPA

Production Realities

Who Should Skip This

References & Further Reading