IaC Drift Detection: Keeping Infrastructure in Sync

Problem Context

You deployed your infrastructure with Terraform three weeks ago. Since then, someone manually adjusted a network security group rule to debug an incident, a colleague added a storage container through the portal, and auto-scaling silently changed a VM SKU. Your Terraform state no longer reflects reality. The next terraform apply may overwrite changes you want to keep — or miss security regressions you need to catch.

Infrastructure driftis the divergence between declared state (your IaC code) and actual state (what's running in the cloud). Drift detection is the practice of continuously measuring that gap and acting on it before it causes an incident.

How Drift Happens

Drift sources fall into three categories:

• Manual changes: Portal edits, CLI one-liners during incidents, direct API calls.
• Automated platform changes: Auto-scaling events, Azure Policy remediations, platform-managed rotations, AKS node pool updates.
• IaC gaps: Resources created outside IaC (the “just this once” resource), properties not managed by the IaC tool (Azure adds new properties your older provider doesn't track).

Terraform: Built-In Drift Detection

Terraform's terraform plan is drift detection — it reads live state, compares it to your configuration, and produces a diff. The key is running it regularly, not just before applies.

Scheduled plan-only runs

# Run plan and capture exit code
# Exit 0 = no changes, Exit 2 = changes detected, Exit 1 = error
terraform plan -detailed-exitcode -out=tfplan

if [ $? -eq 2 ]; then
  echo "DRIFT DETECTED" | notify-slack
fi

GitHub Actions drift detection workflow

name: Drift Detection
on:
  schedule:
    - cron: '0 */6 * * *'   # Every 6 hours
  workflow_dispatch:

jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3

      - name: Terraform Init
        run: terraform init
        env:
          ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
          ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
          ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
          ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}

      - name: Check for Drift
        id: plan
        run: terraform plan -detailed-exitcode -no-color 2>&1
        continue-on-error: true

      - name: Alert on Drift
        if: steps.plan.outputs.exitcode == 2
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            { "text": "⚠️ Infrastructure drift detected in production. Run: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" }

terraform refresh and State Reconciliation

terraform refresh updates the state file to match the real infrastructure withoutmodifying the infrastructure. It's useful when platform-managed changes (like auto-scaling) cause spurious drift noise. However, it also silently accepts manual security changes into state — use it carefully.

# Refresh state from live infrastructure
terraform refresh

# Or during plan (refresh is implicit but can be disabled)
terraform plan -refresh=false   # Skip refresh, use cached state
terraform plan -refresh=true    # Default: refresh before planning

Azure Policy for Continuous Compliance

For Azure-specific deployments, Azure Policy provides a complementary layer. While Terraform detects drift in what IaC manages, Azure Policy detects compliance violations across all resources, including those outside IaC scope.

# Check compliance for a policy assignment
az policy state list \
  --resource-group rg-production \
  --filter "complianceState eq 'NonCompliant'" \
  --query "[].{resource: resourceId, policy: policyDefinitionId}" \
  --output table

# Trigger an on-demand compliance evaluation
az policy state trigger-scan --resource-group rg-production

Drift in Bicep / ARM Deployments

ARM deployments in Incremental mode will not detect or revert manual changes to properties not in your template. Complete mode deletes unlisted resources, effectively enforcing desired state. For continuous drift correction:

# What-if to see drift
az deployment group what-if \
  --mode Complete \
  --resource-group rg-production \
  --template-file main.bicep \
  --parameters @main.prod.bicepparam

# Complete mode deployment to correct drift
az deployment group create \
  --mode Complete \
  --resource-group rg-production \
  --template-file main.bicep \
  --parameters @main.prod.bicepparam

Drift vs Desired State Enforcement

Detection tells you drift exists. Enforcement corrects it. These are two distinct operations with different risk profiles:

• Detection only (safe): Schedule terraform plan / what-if on a cron, alert the team. Humans decide whether to revert or adopt the change.
• Auto-remediation (higher risk): Pipeline automatically runs terraform apply or complete-mode ARM deploy on drift detection. Can overwrite legitimate emergency changes. Use only when you have strong change controls and rollback mechanisms.

For most production environments, detection + human approval is the right balance.

Production Checklist

• Run terraform plan -detailed-exitcode on a schedule (not just before applies)
• Alert the on-call team immediately on drift detection
• Document which resources are intentionally not managed by IaC (e.g., manually scaled resources)
• Use lifecycle { ignore_changes } for properties that legitimately drift (auto-scaling)
• Separate drift detection from remediation — require human approval before auto-apply
• Combine Terraform drift detection with Azure Policy compliance scans for full coverage