# Describing Reality: How We Moved AWS, GCP and Cloudflare to OpenTofu > Our payments and billing infrastructure ran on three clouds and almost none of it was in code. Here's how we imported roughly two thousand live resources into OpenTofu without changing any of them, retired a stale CDK deployment, and the silent traps we hit on the way. - **Author**: Ayush Agarwal - **Published**: 2026-09-23 - **Category**: Infrastructure - **URL**: https://dodopayments.com/blogs/aws-gcp-cloudflare-opentofu --- In August we tried to answer what should have been an easy question: what exactly is running in our cloud accounts? Nobody could answer it from code. Not one resource in our main GCP project was under version control. Kubernetes clusters, databases, networks, firewall rules, WAF policies, data pipelines and backups had all been created by hand in the console. Our AWS account had two CloudFormation stacks from a CDK app, and the larger one hadn't been deployed in two years. It still believed it owned a Kubernetes cluster that had been deleted and rebuilt by hand under a different name. Our Cloudflare account, which sits in front of every request to dodopayments.com, was a dashboard plus ad-hoc `wrangler deploy` runs. At Dodo Payments we run payments and billing infrastructure for merchants around the world. "What changed in production, who changed it, and did anyone review it?" is the first thing an auditor asks and the last thing every incident review lands on. We had already moved Kubernetes to GitOps, but the layer underneath it - networks, databases, IAM, DNS, WAF - had no reviewed, version-controlled description. Five weeks later, all three clouds are described in OpenTofu in one repository: 51 state files and roughly two thousand resources. A nightly job plans every state against production and treats any difference as a bug. Every change is a pull request with a plan attached. Getting there changed nothing that was running. This post covers the method, the numbers and the traps, and what day two looks like now: a drift fix, a provider upgrade or a security patch is a pull request Claude can open. Most of the traps were silent: a tool reported success and was wrong. If you're about to import an estate someone built by hand, those are the parts worth stealing. ## Three Clouds, Zero Sources of Truth ```text Where Production Was Defined, August 2026 ═══════════════════════════════════════════════════════════════════════════ GCP AWS Cloudflare ┌──────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐ │ console clicks │ │ console clicks │ │ dashboard clicks │ │ │ │ CDK stacks, never │ │ ad-hoc wrangler │ │ │ │ updated after day 1 │ │ Pages, cert-manager │ │ │ │ │ │ another repo's TF │ ├──────────────────┤ ├─────────────────────┤ ├─────────────────────┤ │ SOURCE OF TRUTH: │ │ SOURCE OF TRUTH: │ │ SOURCE OF TRUTH: │ │ the console │ │ CloudFormation's │ │ whoever wrote last │ │ │ │ memory (stale) │ │ │ └──────────────────┘ └─────────────────────┘ └─────────────────────┘ Version-controlled description of any of it: none. ``` Each cloud was broken in its own way, and that shaped the work more than anything they shared. - **GCP was entirely hand-built.** No IaC tool had ever touched it, so there was nothing to fight and nothing to read. The only description of the project was the project. - **AWS had a tool that claimed ownership and was wrong.** About 620 objects were hand-built. Another 565 were owned by CloudFormation, most of them by two CDK stacks whose idea of what they owned had drifted a long way from reality. - **Cloudflare had too many owners.** App repos deploy Workers with `wrangler`, Pages deploys itself, cert-manager writes DNS challenge records, and another internal repository applies its own Terraform to the same account. ### Problem #1: Nobody Knew How Big the Problem Was Our first GCP plan, built from targeted `gcloud` queries, counted 1,283 resources and estimated about 570 were importable. Two drafts in, we found we'd missed roughly 70 resources across five whole service categories. A full Cloud Asset Inventory census then reported 17,595 assets. AWS repeated the lesson. The first sizing said 620 hand-built objects. But security group rules are one resource block each, and there were about 250 of them. A single S3 bucket's configuration is split across up to 16 separate resources in the v6 AWS provider. The real figure was 900 to 1,000 blocks, so anyone budgeting from 620 would have been wrong by a third. ### Problem #2: The Tool That Claimed Ownership Was Lying CloudFormation's drift detection reported 14 resources in the larger stack as deleted. When we resolved every stack member against the live AWS API instead, the real number was 36. The drift report stops at 100 rows. It leaves out resource types it can't drift-detect. And `IN_SYNC` only means "the properties I compared match", which says nothing about whether the thing exists. The stack's EKS cluster and all five of its node groups appeared at no status at all. None of them existed. A stale owner is more dangerous than no owner. Any stack operation that touched those phantoms would either fail and roll back the whole stack, or try to recreate databases and compute that had long since been replaced. On payments and billing infrastructure. ## The Mental Shift: Describe First, Change Later The usual way to adopt infrastructure-as-code is to write HCL for the next thing you build and promise to migrate the old stuff later. Later never comes, and you end up with two systems: the code, and everything the code doesn't know about. We flipped it. **The first programme changed nothing.** No fixes, no deletions, no clean-up. Its only output was an accurate, machine-checkable description of what already existed. If a firewall rule was wrong, we described the wrong firewall rule, filed an issue, and fixed it later in its own PR, where the fix would be a reviewable diff instead of a console click. ```text Old Mental Model: IaC Builds Infrastructure ═══════════════════════════════════════════════════════════════════════════ new resource ──► write HCL ──► tofu apply ──► lives in code old resource ──► "we'll migrate it later" ──► lives in the console, forever New Mental Model: IaC Describes Infrastructure ═══════════════════════════════════════════════════════════════════════════ live account ──► import + generate ──► iterate ──► tofu plan: No changes ▲ │ │ ▼ └──── nightly plan against production ◄────────── state = reality (any diff is a bug in the config) ``` Since we couldn't change reality to match the code, the code had to absorb reality as it was. `ignore_changes` covers what a cloud changes on its own schedule. Objects owned by another controller weren't imported at all, and we wrote down why. The definition of done was one sentence, the same for all three clouds: > `tofu plan` reports **no changes**, on every state, on two consecutive nightly runs. A single empty plan is a snapshot. Two in a row show the description survives the provider changing things underneath it. We bent that rule twice, and wrote both down. GCP was signed off on two back-to-back plans from a clean checkout, because the nightly scanner meant to confirm it still had bugs we hadn't found yet. On Cloudflare we waived the wait between four of the nine waves, and the migration record says `WAIVED` for those rows rather than passing them. The nightly scan itself kept running throughout. The deliverable was **100% classification**. Every object ends up in exactly one of two places: described in HCL, or listed in an `UNMANAGED.md` file with an owner and a reason. That's what made the census tractable: ```text What "Everything in GCP" Actually Meant ═══════════════════════════════════════════════════════════════════════════ Cloud Asset Inventory 17,595 assets ├── container image layers 10,200 not configuration ├── Kubernetes objects 5,797 owned by ArgoCD ├── generated backup artefacts 543 owned by the service ├── enabled APIs 110 handled separately │ └── configuration surface 945 ├── described in OpenTofu 488 └── classified in UNMANAGED.md, with a reason 457 ``` "Describe 17,595 things" is impossible. "Describe 945 things, and write down why the rest belong to someone else" is a project. ## Why OpenTofu The OpenTofu vs Terraform debate is usually argued about licences. In August 2023 HashiCorp moved Terraform from MPL 2.0 to the Business Source License, starting with 1.6, and OpenTofu forked under the Linux Foundation a few weeks later. IBM has owned HashiCorp since February 2025. The BSL restricts building competing products, not running your own infrastructure, so it was never a legal blocker for us. What decided it was a feature: **state encryption.** State files hold every attribute of every resource. Both tools write values marked `sensitive` into state in plaintext; the flag only hides them from CLI output. GCS and S3 encrypt objects at rest, but anyone who can read the object reads the plaintext. OpenTofu encrypts state and plan files client-side, before they leave the machine, so reading our state takes two separate grants: the bucket, and decrypt on a KMS key. Terraform's CLI has no equivalent. For payments and billing infrastructure, that settled it. The gap is narrower than it used to be. Terraform 1.10 and 1.11 added ephemeral resources and write-only arguments, which keep a specific secret out of state entirely, and OpenTofu added both in 1.11. For a database password they're the better tool. But they work one attribute at a time, and IP ranges, service account emails and the shape of the whole network still land in state. Terraform wins elsewhere. HCP Terraform gives you managed runs, Stacks and Sentinel policies, you can buy commercial support, and most of the examples you'll find online are written for `terraform`. If you already pay for HCP Terraform, staying put is a reasonable call. ```text OpenTofu vs Terraform, Where It Mattered for Us ═══════════════════════════════════════════════════════════════════════════ OpenTofu Terraform CLI Licence MPL 2.0 BSL 1.1 (since 1.6) Roadmap set by Linux Foundation HashiCorp / IBM Client-side state/plan yes (1.7+) no, backend at-rest only encryption Ephemeral / write-only yes (1.11+) yes (1.10 / 1.11) import blocks, providers, same same HCL, native tests Managed platform bring your own CI HCP Terraform, Stacks ``` Of the features only OpenTofu has, state encryption is the only one we use. No provider `for_each`, no `-exclude`, no OCI registry. That keeps our HCL portable, but the encryption itself is a one-way door: Terraform can't read an encrypted state, so going back means first migrating every state to plaintext through OpenTofu's `unencrypted` method. We took that trade knowingly. Each cloud has one encryption config, backed by that cloud's KMS and loaded into the `TF_ENCRYPTION` environment variable by the Makefile and every workflow: ```hcl # One file per cloud, loaded into TF_ENCRYPTION before `tofu init`. key_provider "gcp_kms" "state" { kms_encryption_key = "projects//locations//keyRings//cryptoKeys/" key_length = 32 } method "aes_gcm" "state" { keys = key_provider.gcp_kms.state } state { method = method.aes_gcm.state enforced = true # refuse to read or write unencrypted state } plan { method = method.aes_gcm.state enforced = true } ``` There's one sharp edge. With `enforced = true`, reading an existing state without the variable fails closed. But if `TF_ENCRYPTION` is unset while you create a *new* state, there's no encryption config at all, and OpenTofu writes it in plaintext. Policy tools read HCL and can't see an environment variable, so no static check will catch this. It's the first rule of our CI contract instead: every entry point exports the right cloud's file before `tofu init`. ## Layer 1: Safety Enforced by IAM The uncomfortable fact about importing: **`import` blocks are executed by `tofu apply`.** You can't populate state without running apply, so "we never run apply during the migration" is a promise nobody can keep. The promise we could keep was that **the identity running apply couldn't change infrastructure.** During the describe phase, the GCP CI identity held two grants: read-only viewer on the project, and object admin on the state bucket, which lives in a separate project. The AWS roles started just as narrow. A malformed config couldn't damage anything, because the cloud API rejected the write. IAM lockout, a forced cluster replacement, a database restart: the failure modes that make migrations scary were off the table. Every layer on top of that assumes the config is wrong until proven otherwise: ```text Why an Import Wave Cannot Break Production ═══════════════════════════════════════════════════════════════════════════ ┌────────────────────────────────────────────────────────────┐ │ Describe-phase CI identity: read-only on the cloud, │ │ write only on its own state bucket │ │ └─► "a wrong config is rejected by the cloud API" │ ├────────────────────────────────────────────────────────────┤ │ Plan gate on every pull request │ │ └─► "a non-empty plan fails the build" │ ├────────────────────────────────────────────────────────────┤ │ Import-only gate on the saved plan │ │ └─► "every change is a no-op carrying an import record" │ ├────────────────────────────────────────────────────────────┤ │ Apply the SAVED plan, behind a production approval │ │ └─► "what runs is exactly what was reviewed" │ └────────────────────────────────────────────────────────────┘ ``` The import-only gate is small and does one thing: it reads the plan OpenTofu actually produced and refuses anything that isn't a no-op carrying an import record: ```python # Abridged from our import-only gate. Input: `tofu show -json wave.tfplan` imports, offenders = [], [] for rc in plan.get("resource_changes", []): change = rc.get("change") or {} actions = change.get("actions") or [] if actions == ["no-op"] and change.get("importing"): imports.append(rc["address"]) elif actions != ["no-op"]: offenders.append((rc["address"], "/".join(actions))) if offenders: for addr, act in sorted(offenders): print(f" {act:10} {addr}", file=sys.stderr) print("Fix the CONFIG to match reality. Never the reverse. Do not apply.", file=sys.stderr) sys.exit(1) ``` The real gate also refuses input that doesn't look like a plan document. An empty JSON object has no `resource_changes`, and "found no changes" is indistinguishable from "read the wrong file". That's the one direction a safety gate must never fail in. The saved plan is what gets applied, never a fresh one: ```bash tofu plan -input=false -out=wave.tfplan tofu show -json wave.tfplan > wave.json python3 import-only-gate.py wave.json # exit 0 => nothing but imports tofu apply -input=false wave.tfplan # the SAVED plan ``` A second `tofu plan` is a second refresh against a live account, and whatever it decides to do was never checked by the gate. ## Layer 2: Generate Config From Reality Every resource block in the repository started as machine-generated config. That was about safety more than speed: `-generate-config-out` emits every attribute the API returns, including ones a cloud CLI's `describe` output hides. A hand-written draft looks right, plans a "small" change to a production resource because it left out a field, and gets approved because the change looks small. The mechanics are two OpenTofu features. An `import` block pairs an address with a cloud ID, and `-generate-config-out` writes HCL for every import that has no configuration yet: ```hcl # imports.tf - transitional, deleted once the wave applies import { to = google_compute_firewall.allow_internal id = "projects//global/firewalls/allow-internal" } ``` ```bash tofu plan -generate-config-out=generated.tf ``` A whole wave looks like this: ```text One Import Wave, End to End ═══════════════════════════════════════════════════════════════════════════ ┌─────────────────────────────────┐ │ Enumerate the LIVE account │ cloud APIs, not docs, └────────────────┬────────────────┘ tags or memory ▼ ┌─────────────────────────────────┐ │ Generate import blocks │ one block per resource ID └────────────────┬────────────────┘ ▼ ┌─────────────────────────────────┐ │ tofu plan │ every attribute the API │ -generate-config-out │ returns, including hidden ones └────────────────┬────────────────┘ ▼ ┌─────────────────────────────────┐ │ Prune the generated HCL │ empty lists, empty strings, └────────────────┬────────────────┘ provider-specific traps ▼ ┌─────────────────────────────────┐ │ tofu plan, and iterate until: │ anything else means the │ N to import, 0 to change │ CONFIG is wrong: fix, repeat └────────────────┬────────────────┘ ▼ ┌─────────────────────────────────┐ │ Import-only gate on the │ refuses any create, │ saved plan + human approval │ update or delete └────────────────┬────────────────┘ ▼ ┌─────────────────────────────────┐ │ tofu apply │ "N imported, 0 added, │ then delete imports.tf │ 0 changed, 0 destroyed" └─────────────────────────────────┘ ``` Generated config is a first draft, and it's wrong in predictable ways. On firewall rules it writes `target_tags = []` and `source_service_accounts = []`, and the Google provider rejects that pair with "Conflicting configuration arguments" even though both are empty. It writes `description = ""` for attributes the API never returned, which several providers refuse. A small prune script removes both, then we iterate until the plan reads `N to import, 0 to add, 0 to change, 0 to destroy`. We looked at bulk-export tools like Terraformer and passed: it had been archived, with unfixed bugs on exactly the GKE resources we needed. The subtler problem is that generation snapshots values another system owns. One production GKE node pool was imported with whatever `node_count` it had that day. The cluster autoscaler owns that number, so as soon as it moved, the nightly scan reported the same drift every night, and running apply would have resized a production pool to match a number in a tfvars file. The fix was to delete the attribute: ```hcl resource "google_container_node_pool" "pools" { for_each = var.node_pools # ... version = each.value.version initial_node_count = each.value.initial_node_count # node_count is deliberately NOT set. Every pool autoscales, so the cluster # autoscaler owns the number. Stating it starts a fight with the autoscaler. lifecycle { # GKE auto-upgrade owns these. ignore_changes = [version, node_config[0].node_image_config] } } ``` That works because of how the provider schema marks the attribute. If an attribute is **Optional and Computed**, leaving it out means "keep whatever the API reports", so it can never diff. If it's **Optional only**, leaving it out means "set it to null", and the next plan wants to change it. We hit that one on Cloudflare, where omitting a D1 database's `read_replication` planned a change from `disabled` to null. Before you delete a generated attribute, check which kind it is. Before you keep one, ask who else writes it. One more trap, because nothing will ever warn you about it: **a spent `import` block is silent.** Once applied, it plans `No changes` at exit code 0 forever. So `imports.tf` is deleted once its wave applies, and `find . -name imports.tf` printing nothing is part of every exit criterion. ## Layer 3: Small States, Keys Derived From Paths The repository holds 51 states: 21 for GCP, 24 for AWS and 6 for Cloudflare, split by layer and environment. Plans stay fast, a mistake has a bounded blast radius, and a PR only plans the states it touches plus any state that calls a module it changed. Nothing flows between states through `terraform_remote_state`, because reading one state's outputs exposes its entire contents to anyone who can read the other. The backend key for every state **is its directory path in the repository**, computed at init and written down nowhere: ```bash p="$(git rev-parse --show-prefix)" # e.g. gcp/environments/prod// tofu init -backend-config="prefix=${p%/}" # GCS backends tofu init -backend-config="key=${p%/}/terraform.tfstate" # S3 backend ``` Nobody types a state key, so nobody types one wrong, and a guard fails any config that hard-codes one. The `${p%/}` matters: `--show-prefix` returns a trailing slash, and `prefix=a/b/` addresses `a/b//default.tfstate`, a different and empty object. A plan against an empty state reports every resource as a create, which is exactly what a tired reviewer might approve as "the import wave". So the plan workflows check that the state object exists before they run `tofu init`. ## AWS: Retiring a Stale CDK Deployment Hand-built AWS objects went the same way as GCP. The part we'd been dreading was getting two CDK-managed CloudFormation stacks out of the account without destroying anything they held. First we checked the assumption everything rested on: nobody was still running `cdk deploy`. The larger stack hadn't been touched since September 2024, and the application now shipped through ArgoCD. That's the only reason a period of dual ownership was survivable. The textbook procedure is: set `DeletionPolicy: Retain` on every resource, deploy that, import everything into the new tool, delete the stack. The Retain edit is supposed to be metadata-only. We took the deployed template verbatim, set Retain on all 331 resources in the larger stack, changed nothing else, and previewed it as a change set. CloudFormation wanted to push property changes onto 149 live production resources and replace 2 outright. Every phantom received a `Modify`. Two years of drift underneath the template meant cross-resource references now resolved differently than when it was deployed. So we reversed the order: ```text Retiring a Stale CDK Deployment ═══════════════════════════════════════════════════════════════════════════ PLAN A: Retain first, then import, then delete preview the "metadata-only" Retain change set │ ├──► 149 live resources get property changes <- not metadata-only ├──► 2 live resources get REPLACED └──► every phantom resource gets a Modify ╳ rejected PLAN B: import first, neutralise, delete last import 431 stack members into 18 OpenTofu states │ <- CloudFormation is no longer the ▼ record of anything we consult termination protection + deny-all stack policy │ ▼ canary 1 ╳ rolled back stack policy is invisible to change sets canary 2 ╳ rolled back UserData read an attribute of a phantom │ ▼ prune verified phantoms, Retain every survivor, verify LIVE template │ ▼ delete: 455 resources retained, 0 failures, every state "No changes" ``` **Import first.** All 431 importable stack members went into 18 OpenTofu states, each wave proved import-only by the gate. From then on OpenTofu was the only record we consulted, so CloudFormation had already lost its authority in practice. Deleting the stacks just made that official. **Neutralise.** Termination protection blocks deletion. A deny-all stack policy blocks updates. They're different controls, and it's easy to set the second and believe you've done the first. **Then delete, carefully.** The first canary teardown was denied at execute time by the deny-all policy we'd forgotten to relax. A stack policy is invisible to change set computation, so a change set that passed every check still failed. The second attempt updated 297 resources cleanly, then hit this and rolled everything back: ```text Unable to retrieve PublicIp attribute -- instance i-... does not exist ``` One instance's `UserData` read the public IP of another instance, a phantom. We had proved no change could cause a replacement, which was true and beside the point. CloudFormation resolves values like that with a live AWS call at execute time, and the call can't succeed against something that's gone. Both canaries ran on the smaller stack, and both failed closed, which we verified instead of assuming: zero deletions, the live template byte-identical, every OpenTofu state still planning `No changes`. Don't count on that. On the larger stack, one attempt failed and then failed to roll back, landing in `UPDATE_ROLLBACK_FAILED`, which accepts no operation until someone recovers it by hand. What finally worked was explicit: prune every verified phantom from the template, apply `Retain` to every survivor, confirm it in the *live* template rather than the change set we submitted, and only then delete. Across both stacks, 455 resources came back `DELETE_SKIPPED` with zero failures, and every OpenTofu state planned `No changes` afterwards. Then we removed the CDK toolkit itself from every enabled region. Two more AWS lessons apply well beyond CDK: - **An `import` block ignores the per-resource `region` attribute** and runs against the provider's region. With all our SES resources in one state, objects that existed only in the other region failed loudly. The two that existed in both regions quietly imported the wrong region's object and **reported success**. We keep one state per region now. - **A resource no state describes produces no diff.** `tofu plan` stays clean whether or not it exists. A tag-based census put hand-built launch templates at 1 of 11. The real number was 7, because seven of them had no tags at all. So each cloud's final reconciliation enumerated every live object and compared IDs against every state file, instead of reading plans. ## Cloudflare: Five Writers, One Account Cloudflare fronts everything: every zone and DNS record, WAF and rate-limit rules, redirects, access policies, and the storage behind our Workers. It needed three decisions the other clouds didn't. **State can't live in the account it describes.** R2 is inside the same account, and a description that depends on the account being healthy disappears exactly when you need it. Cloudflare state lives on GCS, in its own bucket with its own key and service account, so neither cloud's pipeline can read the other's state. **Ownership is drawn per object.** On GCP and AWS, "describe everything you can see" is a reasonable default. On Cloudflare it's wrong: ```text Five Writers, One Cloudflare Account ═══════════════════════════════════════════════════════════════════════════ Writer Owns Excluded here by ───────────────────────────────────────────────────────────────────────── OpenTofu (this repo) zones, DNS, WAF, rulesets, - settings, storage, Access wrangler (app repos) Worker scripts, bindings, resource type routes, custom domains (policy deny list) Cloudflare Pages projects, deployments resource type cert-manager (in GKE) _acme-challenge TXT records record name another internal repo its own KV, R2, D1, Queues object ID ``` The other repository's objects are excluded by ID, because its names come from its own CI settings and a prefix can't be trusted. Workers custom domains synthesise read-only DNS records, which the generator filters out before anything becomes HCL. Types owned by `wrangler` and Pages are on a policy deny list, so describing a Worker script here fails validation instead of starting a tug-of-war on the next deploy. `cloudflare_zone_dnssec` is on the list too: importing it has been known to switch DNSSEC on. **The API has a hard ceiling.** Cloudflare allows 1,200 API requests per five minutes per user, counted across the dashboard and every token, and going over blocks all API calls for the next five minutes. The v5 provider doesn't back off on a 429, so Cloudflare plans run one at a time while GCP and AWS fan out in parallel. Cloudflare has no federated credential issuance, so its jobs use narrowly scoped, expiring API tokens. The trap that nearly got us was the `for_each` key for DNS records. The list API doesn't return records in a stable order, so keying by list position was out: a key that moves is a destroy-then-create of a live record. Our first fix sorted records by content within each name and type, and keyed them by rank. That was stable across repeated runs, which is all our round-trip test checked. Then we measured an insert. One name carried 22 TXT verification records, and adding a 23rd regenerated as 22 in-place `content` updates plus a create. A rank describes a record's neighbours, so every sibling after the new one shifted. Now the key comes only from the record's own fields, which also means the module can verify it: ```hcl variable "records" { # Keyed by "//". type = map(object({ name = string type = string content = string ttl = number proxied = bool })) validation { condition = alltrue([ for k, v in var.records : k == "${v.name}/${v.type}/${substr(sha256(v.content), 0, 12)}" ]) error_message = "Each key must be derived from its own record's name, type and content." } } ``` An insert is now one create and nothing else moves, and a key that disagrees with its own record fails at plan time. The cost: editing a record's content in place gives it a new key, so it plans as destroy-then-create instead of an update. For verification TXT records, MX targets and CNAMEs, adding and removing is the common operation, so we took that trade. Maps like this carry their own risk. Our parked domains are one map, and deleting one line from a tfvars file destroys a zone along with its records, settings and redirects, while the diff reads as a one-line removal. So a check refuses any plan that destroys a zone or an R2 bucket. A skimmed review won't catch that class of accident, so it needs a deliberate bypass. The Cloudflare migration ran in nine waves over three days. Every wave that imported anything applied as `N imported, 0 added, 0 changed, 0 destroyed`, and the account is now 794 described objects across six states, with everything else classified by owner. ## One Repository, Three Clouds GCP and AWS started as separate repositories, so that no CI identity or plan gate spanned both clouds. By the time AWS reached parity, both carried copies of the same eight scripts and three policy files, and **all eleven had diverged.** A fix landing in one wasn't landing in the other. So we merged them and kept the isolation in the design: - **Cloud is the first path segment.** Each cloud directory owns its bootstrap, environments, modules, policies and docs. Only cloud-agnostic tooling lives at the root. - **No job ever holds two clouds' credentials.** Validation needs none. Plan, apply and drift are split per cloud, each with its own federated identity, and nothing crosses the boundary: no remote state, no cross-cloud data sources, no shared modules. - **One plan gate per cloud**, so a broken AWS plan doesn't block a GCP-only pull request. ```text infra/ (conceptual) ├── gcp/ aws/ cloudflare/ same shape inside each: │ bootstrap/ state bucket, key, CI identity │ environments/// one state per directory │ modules/ that cloud's provider only │ policy/ Conftest rules for that provider │ UNMANAGED.md every exclusion, with owner and reason ├── scripts/ gates that run over every tree └── .github/workflows/ validate - plan-* - apply-* - drift-* ``` Because keys are paths and every path gained a cloud prefix, 45 state objects had to move. OpenTofu encrypts state before it reaches the bucket, so the objects are portable byte for byte: a server-side copy keeps lineage and serial and can be verified by hash, which `tofu init -migrate-state` can't offer. All 45 states had to plan clean at their new keys before the PR opened, and the old objects stayed in place as the rollback. ## Layer 4: Guardrails on Every Pull Request ```text A Change, From Pull Request to Production ═══════════════════════════════════════════════════════════════════════════ pull request │ ├──► validate (no credentials): fmt, validate, guard tests, Conftest, │ Trivy, TFLint, gitleaks, dup-check, workflow lint ──► gate │ ├──► plan-gcp only the states this PR touches ──► gcp-plan-gate ├──► plan-aws (or that call a changed module), ──► aws-plan-gate └──► plan-cloudflare destroy table in the PR summary ──► cloudflare-plan-gate │ ▼ review, merge to main │ ▼ apply-: re-plan from scratch, print every destroy, wait for the `production` approval, apply │ ▼ drift-, nightly: plan every state; drift opens or updates one issue per cloud, closed after two clean nights ``` The part that has paid for itself most is policy-as-code with Conftest. Each rule encodes a failure mode we've decided to make impossible. This one exists because the authoritative IAM resources are the easiest way to lock yourself out of a GCP project: ```text # AUTHORITATIVE: replaces all bindings rather than adding to them, so one built # from incomplete config removes every binding it omits, Google's own service # agents and your own access included. Use the purely additive _iam_member. deny contains msg if { some t in resource_types startswith(t, "google_") endswith(t, "_iam_policy") msg := sprintf("%s is AUTHORITATIVE. Use the _iam_member equivalent.", [t]) } ``` Other rules refuse long-lived service account keys, refuse the Kubernetes and Helm providers (Kubernetes objects belong to ArgoCD), require exact provider pins that agree across a cloud's states, and refuse the same resource body appearing in two states. Every rule has tests, including the cases where it must refuse: our first rules matched a key the HCL parser never populates, and passed against every file we had. Once the describe phase ended and the CI identities gained write access, the pull request became the only sanctioned way anything changes. The plan workflow lists every resource that will be destroyed in the PR summary. On merge, apply re-plans from scratch so a stale plan can't be swapped in, prints the destroy list again, and waits for a `production` environment approval. An `authorize` job re-checks the actor and branch in code, because environment settings don't show up in code review. ## Coding Agents Did the Toil, the Contract Kept Them Honest Much of this work is toil, and a lot of it was done by coding agents working in the repository: roughly a quarter of the commits carry an agent as co-author. Every cloud directory has an `AGENTS.md` stating the rules for humans and agents alike, and the root one wins on conflict. It worked because the definition of done is binary and machine-checkable. An agent can't declare victory on an import wave: `tofu plan` either says `No changes` or it doesn't. The read-only identity meant a wrong config couldn't hurt anything while it was being iterated, and every change still landed through the same PR, plan gate and approval as anyone else's. The speed was real, and it's easy to overstate. The GCP plan was written on August 14 and budgeted 14 to 18 weeks of engineering time. The parity sign-off merged the next evening. The AWS plan was written on September 15, and three days later every AWS state planned clean and CDK was gone. But parity turned out to be where the work started. The following weeks went into policy, linting, write access and the drift scanner, and once the scanner worked it kept finding places where our exact description wasn't. ```text Programme Timeline, 2026 ═══════════════════════════════════════════════════════════════════════════ Aug 14 GCP plan written budget: 14-18 weeks Aug 15 GCP parity sign-off, eleven states Aug 16 DNS records the first pass missed; 488 described, 457 classified Aug 17 ─┐ │ policy-as-code, linting, lockfiles, drift-scanner fixes, Sep 14 ─┘ write access, the first deletions through PRs Sep 15 AWS plan written Sep 18 AWS parity, CDK gone from the account Sep 19 AWS repository folded in, 45 states re-keyed Sep 19 Cloudflare wave 0: tree, policy and state store Sep 21 Cloudflare: last of nine waves applied, parity audit complete ``` ## Day Two: Claude Opens the Pull Request Parity was never the goal. The goal was to make the next thousand changes boring, and that's where this paid off most. Once every resource is described in code, gated by a plan and checked every night, running infrastructure stops being console work and becomes code work. Claude is very good at code work. Day-two changes now follow one loop: an issue comes in, and a pull request goes out, often opened by Claude: ```text Day Two: Where Infrastructure Changes Come From ═══════════════════════════════════════════════════════════════════════════ nightly drift issue ───────┐ security finding ──────────┤ provider or tool upgrade ──┼──► Claude reads AGENTS.md and the issue, "we need a new X" ─────────┘ edits the HCL, opens a pull request │ ▼ same gates as any human: validate, Conftest, Trivy, TFLint, plan gate, and the destroy table in the PR summary │ ▼ human review ──► merge ──► `production` approval ──► apply ``` - **Drift.** Break-glass access to the consoles has to exist, and nightly drift detection is what makes it acceptable. A setting changed by hand is an issue by the next morning, with the plan diff attached. Claude reads the diff and opens a PR that either captures the change in code or reverts it. When someone edited a WAF rule in the Cloudflare dashboard, the fix was a PR that wrote the edit into the config. - **Vulnerabilities.** Our security assessment maps every finding to a GitHub issue, which gives an agent a precise, reviewable brief. A finding like an over-broad role on a service account becomes a one-line HCL change, and the plan shows exactly one IAM binding being destroyed. Narrowing IAM grants, tightening Kubernetes RBAC and turning on audit logging all arrived this way. We also vendor Cloudflare's open-source security-audit skill into the repository at a pinned commit, so every audit an agent runs uses the same methodology, and upgrading that methodology is itself a reviewable diff. - **Upgrades.** Dependabot bumps the providers and the SHA-pinned GitHub Actions every week. Provider pins are exact, so every bump runs the plan gate against production: a provider that changes behaviour fails CI instead of merging quietly, and the fix is an ordinary edit on the same branch. Dependabot can't see the OpenTofu and scanner versions pinned inside workflow files, so a comment lists every place they live, and bumping all of them is one agent PR. This is safe for the same reason the import was. Nothing in the loop needs a credential that can change production: CI plans every PR with a read-only identity, and the only path to apply is a merge followed by a `production` approval. The plan is the review artifact, so the reviewer checks what will change rather than whether the agent understood the ticket. What stays human is intent: whether a console edit should be kept, and whether a resource should really go. Before, patching a finding meant someone with console access clicking through a fix, and nobody could say afterwards exactly what had changed. Now it's a pull request with a plan attached, reviewed like any other change and checked again that night. ## The Results ```text Before vs After ═══════════════════════════════════════════════════════════════════════════ Before After ───────────────────────────────────────────────────────────────────────── Described in code almost nothing ~2,000 resources, 51 states, 3 clouds Source of truth consoles + a stale CDK app one repository Drift detection none nightly, per cloud How a deletion happens a CLI command a PR with a destroy table and approval How a fix or upgrade console clicks a PR, often opened lands by Claude State at rest n/a encrypted client-side with each cloud's KMS CloudFormation / CDK 2 stacks + toolkit 0 Mutations to import it - 0 ``` - **Parity without mutation.** Every import wave across all three clouds applied as imports only: zero resources added, changed or destroyed. - **CDK is gone.** Both stacks, their nested stacks and the CDK toolkit are removed, with 455 resources retained and zero failures. - **Deletion became safe enough to actually do.** Once the description existed, clean-up became a reviewed PR whose plan *is* the destroy manifest. On AWS, two waves destroyed 418 dead resources, including an idle secondary-region environment with no traffic in 90 days. Imported config carries no `depends_on`, so that environment came down in four ordered PRs, one dependency layer at a time. On GCP, a self-hosted PaaS and a handful of idle VMs went the same way: disarm deletion protection in one PR, destroy in the next. - **Drift is visible.** A change made outside the pipeline is an issue by the next morning. - **Maintenance is a pull request.** Drift, security findings and upgrades come in as issues and go out as PRs, many of them opened by Claude and all of them through the same gates. ## Should You Do This? This approach makes sense if: - Your cloud estate was built by hand, or by a tool nobody runs anymore - More than one person changes infrastructure, and "who changed this?" has no reliable answer - You need to show an auditor what changed, when, and who approved it - You run more than one cloud and want one review surface and one set of guardrails - You want deletions to be reviewable instead of heroic It's probably overkill if: - You're starting fresh. Write the IaC as you build; importing is a brownfield tool - Your whole estate is a handful of resources one person fully understands - You won't act on a nightly drift report. An ignored drift issue teaches everyone that red is normal - What you want to manage is Kubernetes objects. Those belong to a GitOps controller The honest answer: describe-first is slower on day one than "just start writing Terraform for new things". It's much faster by week five, because you're never maintaining two worlds. And the constraint that felt most restrictive, changing nothing until the description is exact, is the one that made every later change safe. ## Key Takeaways 1. **Describe before you change.** Import to parity with zero mutations, and turn every fix you're tempted to make along the way into its own PR. 2. **Make an empty plan the contract.** It fails CI by day and opens an issue by night. When you bend the rule, write down that you did. 3. **Enforce safety with IAM, not discipline.** Import blocks run under `apply`, so the identity running it shouldn't be able to change anything. 4. **Generate config from live state, then ask who else writes each attribute.** Omit Optional+Computed values another controller owns; Optional-only ones become null when omitted. 5. **Key `for_each` by the object's own identity.** Positions and sort ranks move when a sibling appears. A content hash doesn't, and HCL can validate it. 6. **Measure, don't estimate.** A resource no state describes produces no diff, so reconcile against live APIs. 7. **Then let agents do the maintenance.** Once every change is a PR with a plan, read-only CI and a human approval, drift fixes, security patches and upgrades are work Claude can open and your gates can check. Infrastructure doesn't have to be a pile of console clicks everyone is afraid to touch. It has to be written down exactly as it is, first, and checked every night against what's actually running. _We're building payments and billing infrastructure at Dodo Payments. If infrastructure-as-code, multi-cloud platforms and making production boring sound interesting, [we're hiring](https://dodopayments.com/careers)._ --- - [More Infrastructure articles](https://dodopayments.com/blogs/category/infrastructure) - [All articles](https://dodopayments.com/blogs)