The complete documentation set — operations, architecture, deployment, roadmap, and decision records — in one page. Every link resolves in-page; no external access required.
The comprehensive reference for running, extending, operating, and troubleshooting IaCTranslate. Read the Summary and Architecture first, then jump to whichever How-To or Troubleshooting section you need. For the why behind the design, see Architecture & Design and the ADRs.
Contents 1. Executive summary 2. Architecture at a glance 3. Repository map 4. Core concepts (the vocabulary) 5. The pipeline, step by step 6. How-to: run it 7. How-to: extend it 8. Configuration reference 9. API reference 10. Testing & CI 11. Deployment & operations 12. Troubleshooting 13. FAQ & glossary 14. Performance 15. Security model 16. Error-handling philosophy
Design rationale (principles, the canonical model, request-flow diagrams, scope, assumptions, "why not …") lives in Architecture & Design and the ADRs. This guide is the operations reference.
What it is. IaCTranslate converts any infrastructure inventory — VMware (RVTools), Microsoft Hyper-V, Kubernetes, a CMDB/spreadsheet export (ServiceNow, Device42, Lansweeper, or hand-rolled), or an existing AWS/Azure fleet — into production-ready Terraform for AWS, Azure, GCP, OCI, or DigitalOcean, and can recommend the best-fit cloud. It never connects to the customer environment; it works entirely from exported inventory files.
The core idea (why it's trustworthy). It is not "an LLM writes Terraform." It's a
deterministic translation layer. The AI (optional) makes only structured decisions
(which application group, which instance size); Python + Jinja2 templates emit the actual
.tf, and a validation layer re-checks every decision. Output is reproducible, auditable,
and — proven by CI — valid against the real cloud providers (tofu validate).
The moat. Competitors are the cloud vendors' own migration tools (AWS Migration Hub, Azure Migrate, Google Migrate). They are single-cloud (lock-in), never emit portable IaC, and will never recommend a rival cloud. IaCTranslate is source-agnostic + cloud-neutral + unbiased, and its generic source ingests any company's CMDB/spreadsheet with no bespoke parser — so it works for every company, not just VMware shops.
Status. CLI + FastAPI + Next.js web UI. On top of the core translator it ships a full
migration-platform layer — assessment, confidence scoring, executive reports, architecture
diagrams, infrastructure diff, brownfield adoption, load balancer topology,
managed-DB re-platforming advice, migration wave planning, a Kubernetes discovery source,
Pulumi/CloudFormation/Bicep/CDK/Kubernetes renderers, a policy engine, an
Infrastructure Graph IR, async jobs + audit, and opt-in GitOps.
~446 tests, 8 green CI jobs (lint, pytest 3.9/3.11/3.12, Docker health, web build, real
Terraform validate). Repo: github.com/Kolanupaka92/iactranslate (public).
INPUT (any inventory) OUTPUT (any cloud)
┌───────────────────────────────┐ ┌──────────────────────────────┐
│ sources/ (Source registry) │ │ targets/ (Target registry) │
│ vmware · hyperv · generic · │ │ aws · azure · gcp │
│ cloud │ │ (catalog + mapping + │
└──────────────┬────────────────┘ │ Jinja2 templates) │
│ raw records └───────────────▲──────────────┘
▼ │
parse → normalize → agents(classify → rightsize → network) → validate → render → package(zip)
│ │ │ │
NormalizedVM provider (rule | anthropic) PlanValidation Terraform files
normalize and render is source- and cloud-agnostic. A Source
only decides where the estate came from; a Target only decides where it's going.sources/ and targets/ are parallel: pick by name or
auto-detect; add a new one without touching the pipeline.iactranslate/
├─ src/iactranslate/
│ ├─ models.py # Pydantic canonical model: NormalizedVM, ComputePlan,
│ │ # NetworkPlan, MigrationPlan, enums (Tier, Environment…)
│ ├─ config.py # Env-driven limits (upload size, VM count, projects, CORS)
│ ├─ normalize.py # raw records → List[NormalizedVM] (unit coercion, dedupe)
│ ├─ pipeline.py # run_pipeline(): the end-to-end orchestrator
│ ├─ recommend.py # deterministic multi-cloud recommender
│ ├─ assessment/ # pre-migration readiness assessment (findings + score + HTML)
│ ├─ packager.py # write project tree + migration-summary.md + assessment + zip
│ ├─ cli.py # `iactranslate translate|recommend|assess`
│ ├─ parsers/ # back-compat shim → sources (legacy parse/detect_format)
│ │
│ ├─ sources/ # ── INPUT registry ──
│ │ ├─ base.py # Source protocol + detection helpers
│ │ ├─ _columns.py # tolerant column matching (find_column, cell)
│ │ ├─ vmware/ # RVTools .xlsx + vSphere .csv
│ │ ├─ hyperv/ # Get-VM export
│ │ ├─ generic/ # ★ any CMDB/spreadsheet (synonym auto-detect + column_map)
│ │ └─ cloud/ # existing AWS/Azure fleet (type→vCPU/mem via catalogs)
│ │
│ ├─ targets/ # ── OUTPUT registry ──
│ │ ├─ base.py # Target protocol + InstanceSpec + smallest_fit
│ │ ├─ aws/ # catalog.py, mapping.py, templates/*.j2 (EC2/VPC/SG)
│ │ ├─ azure/ # (VM/VNet/NSG via azurerm)
│ │ └─ gcp/ # (Compute Engine/VPC/firewalls via google)
│ │
│ ├─ agents/ # classify → rightsize → network
│ │ ├─ __init__.py # build_migration_plan()
│ │ ├─ classifier.py, rightsizing.py, network.py, heuristics.py
│ │ └─ providers/ # rule_engine.py (default) | anthropic_provider.py
│ │
│ ├─ validation/ # validators.py: CIDR/dup/catalog/naming checks
│ ├─ generator/ # renderer.py: MigrationPlan → {filename: content}
│ └─ api/ # main.py (FastAPI) + store.py (in-memory project store)
│
├─ web/ # Next.js UI (App Router, Tailwind) — wizard over the API
├─ tests/ # pytest suite + fixtures + test_e2e.py
├─ scripts/make_fixtures.py# generates the 5 sample inventories
├─ Dockerfile, .dockerignore
├─ .github/workflows/ci.yml
└─ pyproject.toml
| Concept | What it is | Where |
|---|---|---|
| NormalizedVM | The canonical unit of a workload (name, cpu, memory_gib, disks, os, ip, …). Every source produces these; everything downstream consumes them. | models.py |
| MigrationPlan | The validated object the generator renders: network + compute[] + app_groups[] + source_platform + target + region. |
models.py |
| Source | Reads one inventory format → raw records that normalize.py understands. Has detect(path)→confidence and parse(path, column_map). VMware/Hyper-V/CMDB/cloud are tabular; Kubernetes reads kubectl -o json (containers sized from resource requests). |
sources/base.py |
| Target | One cloud: an instance catalog, tier→family/subnet/security mappings, OS→image detection, and Jinja2 templates. | targets/base.py |
| Provider | Makes the structured decisions (grouping, instance choice). rule (deterministic, default) or anthropic (Claude) — reachable via --provider/API provider field/web toggle, not just an env var. Always re-checked by validation; plan.provider_used honestly records which one ran. |
agents/providers/ |
| Narrative | The executive report's AI-written summary paragraph — only when the plan itself was AI-assisted, else a deterministic paragraph from the same facts. Presentation-layer only; never changes the plan. | narrative.py |
| Recommender | Runs all clouds on one inventory and scores cost (0.45) + fit (0.30) + OS-affinity (0.25). 2.0 adds decisiveness (clear/moderate/close), annualized cost, and estate notes. | recommend.py |
| Assessment | Pre-migration read of the estate: risk/cost/data-quality/capacity findings + a 0-100 readiness score. Deterministic, no AI. Emits JSON + a standalone HTML report. | assessment/ |
| Confidence Engine | Scores how sure each decision is (sizing/classification/image/cost) per workload + plan-level, from observable signals. | confidence.py |
| Executive Report | One client-facing HTML page composing plan + cost + assessment + confidence + recommendation + architecture diagram. | exec_report.py |
| Architecture Diagram | Deterministic SVG + Mermaid of the target topology (VPC → subnets → tiered instances → load balancers). | diagram.py |
| Load Balancer | Any (tier, environment, subnet_tier) group with >1 instance gets one, with listeners from the tier's own security-group ingress (AWS ALB, Azure Standard LB, GCP Network/Internal LB, OCI flexible LB). |
agents/network.py, models.py |
| Infrastructure Diff | Drift between two inventory snapshots (added/removed/modified + aggregate deltas). | diff.py |
| Renderer | Swappable IaC output: terraform (default, HCL, all 5 clouds), pulumi (Python, AWS/Azure/GCP — not yet OCI/DigitalOcean), cloudformation (JSON, AWS-only), bicep (Azure-only), cdk (Python, AWS-only), or kubernetes (JSON/KubeVirt, any cloud) — the latter four render from the Infrastructure Graph, not the plan. |
renderers/ |
| Brownfield | Existing cloud fleet with resource ids → Terraform/Pulumi import blocks (adopt, don't recreate). |
sources/cloud, renderers/ |
| Re-platforming advisor | Flags database-tier workloads as managed-DB candidates (RDS/Cloud SQL/Azure SQL) with engine detection + caveats. Advisory-only — never changes the plan. Emits replatforming.json. |
replatform.py |
| Migration wave planner | Sequences workloads by tier dependency (data/cache → app → web) + environment order (dev → staging → prod), with depends_on chains, rollback strategy, validation checks, LB-aware downtime estimates. Advisory-only. Emits waves.json. |
waves.py |
| GitOps | Opt-in CI/CD workflow (plan on PR, apply on merge) + .gitignore, target/renderer-aware. | gitops.py |
| Validation layer | Never trusts provider output: checks catalog membership, CIDR overlap/containment, duplicate names, referential integrity. | validation/validators.py |
The raw-record contract (what a Source.parse returns, consumed by normalize):
name, cpus, memory as memory_mib or (memory_value + memory_unit), disks as
disks_mib or (disk_value + disk_unit), plus optional os, network, ip,
cluster, datacenter, dns_name, powerstate.
run_pipeline() in pipeline.py is the spine. Each stage:
resolve_source(path, name) auto-detects (or honors an
explicit --source); get_target(name) picks the cloud.source.parse(path, column_map) → raw records. (VMware reads RVTools sheets;
Hyper-V converts bytes→MiB; generic maps columns; cloud looks instance types up in the
target catalogs to recover vCPU/mem; Kubernetes reads kubectl -o json and sizes each
workload from its containers' resources.requests, with StatefulSet volume claims as disks.)normalize() coerces units to canonical (MiB→GiB), parses IPs, dedupes by
name → List[NormalizedVM]. Guardrails: empty → error; > MAX_VMS → error.agents/classifier.py). Provider groups VMs into applications, detecting
environment (prod/dev/…) and tier (web/app/database/cache).agents/rightsizing.py + sizing.py). Per VM, effective_demand() decides
the requirement: if the source carries utilization (CPU/mem %), size to actual usage
at IACTRANSLATE_TARGET_UTILIZATION headroom (right-sizing); otherwise size to raw
allocation with 1.2× headroom (unchanged). The provider proposes an instance; the code
re-checks it against the target catalog (falls back to smallest_fit), computes cost,
and derives subnet tier + security group. Right-sized rows record the before/after and
surface in the migration summary and API (right_sized_count). Cost comes from
pricing.monthly_cost(): static catalog rates by default, or live market prices when
IACTRANSLATE_PRICING=live (Azure Retail Prices API needs no credentials; AWS uses boto3
if creds exist; GCP uses the Cloud Billing Catalog when IACTRANSLATE_GCP_BILLING_API_KEY
is set — summing the machine family's vCPU-core + RAM-GB SKUs; anything unavailable falls
back to static). pricing_source (static/live)
is surfaced in the summary and API. The recommender always uses static rates for a fair
apples-to-apples comparison.agents/network.py, deterministic — never the LLM). Allocates VPC/VNet,
public+private subnets per AZ, and the security groups the tiers imply.MigrationPlan (with the real source_platform).assert_valid). Any issue raises PlanValidationError; nothing is written.generator/renderer.py). Loads the target's Jinja2 templates and produces
{filename: content} — real HCL.packager.py). Writes the project tree + documentation/migration-summary.md
and (optionally) a .zip.Output tree (per cloud): versions.tf, provider.tf, variables.tf, terraform.tfvars,
networking.tf, security.tf, loadbalancer.tf, compute.tf, storage.tf, outputs.tf, main.tf,
README.md, graph.json, assessment.json, confidence.json, decisions.json, waves.json,
documentation/migration-summary.md, modules/ — plus replatforming.json when any
database-tier workloads are detected.
| Tool | Version | Required for | Install |
|---|---|---|---|
| Python | 3.9+ (CI tests 3.9 / 3.11 / 3.12) | The core service, CLI, API — required | python.org · pyenv · brew install python@3.12 |
| pip + venv | bundled with Python | Installing the package | (comes with Python) |
| Node.js + npm | Node 20+, npm 10+ | The web UI (web/) only |
nodejs.org · nvm install 20 · brew install node |
| OpenTofu or Terraform | 1.5+ | Validating / deploying the generated .tf, and the tofu validate E2E test |
brew install opentofu · opentofu.org · terraform.io |
| Docker | any recent | Building/running the container (optional) | docker.com |
| gh CLI | any | GitHub / CI operations (optional) | brew install gh |
Python libraries install automatically via pip install -e ".[dev]" — no manual step.
They are: pandas, openpyxl (read .xlsx/.csv), jinja2 (templates), fastapi + uvicorn
+ python-multipart (API), pydantic (models), and the dev/optional extras pytest,
httpx, ruff, anthropic. No system packages beyond Python itself are needed for the
core service; tofu/terraform and node are only for the validation and UI paths above.
Verify your toolchain:
python3 --version # >= 3.9
node --version # >= v20 (only if using the web UI)
tofu version # >= 1.5 (only for validating/deploying output)
cd iactranslate
python3 -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]" # installs the package + all Python deps
python scripts/make_fixtures.py # writes 5 sample inventories to tests/fixtures/
# VMware → AWS, zipped
iactranslate translate tests/fixtures/rvtools_sample.xlsx --target aws --out ./out-aws --zip
# Any source, auto-detected → Azure
iactranslate translate tests/fixtures/hyperv_sample.csv --target azure --out ./out-hv
# A CMDB with non-standard headers → GCP, mapped explicitly
iactranslate translate my-cmdb.csv --source generic \
--map "name=Hostname,cpu=Cores,memory_gib=RAM GB,disk_gib=Storage GB,os=OS" \
--target gcp --out ./out-cmdb
# Pulumi output (AWS) instead of Terraform, with a GitOps CI/CD workflow
iactranslate translate rvtools.xlsx --target aws --renderer pulumi --gitops --out ./out-pl
# CloudFormation output (AWS-only), rendered from the Infrastructure Graph
iactranslate translate rvtools.xlsx --target aws --renderer cloudformation --out ./out-cfn
# Bicep output (Azure-only), also rendered from the Infrastructure Graph
iactranslate translate rvtools.xlsx --target azure --renderer bicep --out ./out-bicep
# AWS CDK (Python) output, also rendered from the Infrastructure Graph
iactranslate translate rvtools.xlsx --target aws --renderer cdk --out ./out-cdk
# Kubernetes/KubeVirt output, works for any target (cloud-agnostic)
iactranslate translate rvtools.xlsx --target gcp --renderer kubernetes --out ./out-k8s
# Read containerized workloads off a cluster (kubectl -o json) as the source
kubectl get deployments,statefulsets -A -o json > k8s.json
iactranslate translate k8s.json --source kubernetes --target aws --out ./out-from-k8s
Flags: --target aws|azure|gcp|oci|digitalocean, --source auto|vmware|hyperv|kubernetes|generic|cloud, --map,
--region, --name, --zip, --renderer terraform|pulumi|cloudformation|bicep|cdk|kubernetes
(CloudFormation and CDK are AWS-only; Bicep is Azure-only; Kubernetes has no
target restriction), --gitops (adds .github/workflows/* + .gitignore).
Every generated project also ships, under documentation/, an executive report
(executive-report.html), an architecture diagram (architecture.svg/.md), the
assessment (assessment.json, assessment.html), and confidence.json. A brownfield
export (a cloud fleet CSV with resource ids) additionally yields imports.tf /
Pulumi import options so existing infra is adopted, not recreated.
iactranslate recommend tests/fixtures/rvtools_sample.xlsx
# prints a ranked table (score, $/mo, cost/fit/OS) + per-cloud rationale
iactranslate assess tests/fixtures/rvtools_sample.xlsx
# prints a readiness score (0-100) + categorized findings (risk/cost/data-quality/capacity)
iactranslate assess my-cmdb.csv --json # machine-readable
iactranslate assess my-cmdb.csv --html-out report.html # standalone client report
A translate run also writes the assessment into the project package
(assessment.json + documentation/assessment.html).
# Client-facing executive report (HTML): plan + cost + assessment + confidence + recommendation + diagram
iactranslate report rvtools.xlsx --target aws --out report.html
iactranslate report rvtools.xlsx --no-recommend # skip the 3-cloud compare
# Drift between two inventory snapshots (added/removed/resized + aggregate deltas)
iactranslate diff old-inventory.csv new-inventory.csv
iactranslate diff old.csv new.csv --json
Enforce organization rules on the plan before rendering. deny violations abort
the run; warn violations are reported (policy-report.json) but don't block.
# policy.json — activate + parameterize built-in rules
# { "no_public_subnets": {}, "allowed_instance_families": {"families": ["t3","m5"]},
# "max_vcpu": {"max": 16}, "max_monthly_cost": {"budget_usd": 5000},
# "naming_prefix": {"prefix": "acme_", "severity": "warn"}, "require_nat": {} }
iactranslate translate rvtools.xlsx --target aws --policy policy.json --out ./out
Built-in policies: no_public_subnets, allowed_instance_families, max_vcpu,
max_monthly_cost, naming_prefix, require_nat. Any policy's severity can be
overridden in its config ("severity": "warn"|"deny"). Policies are read-only —
they never mutate the plan (see Architecture › Policy engine).
Over the API: pass policy on project create; GET /policies lists the rules.
uvicorn iactranslate.api.main:app --port 8000 # add --reload for dev
# with a frontend:
IACTRANSLATE_CORS_ORIGINS=http://localhost:3000 uvicorn iactranslate.api.main:app
Full curl walkthrough:
PID=$(curl -s -X POST localhost:8000/projects -H 'content-type: application/json' \
-d '{"name":"demo","target":"aws","source":"auto"}' | python -c 'import sys,json;print(json.load(sys.stdin)["id"])')
curl -s -X POST localhost:8000/projects/$PID/upload -F "file=@tests/fixtures/rvtools_sample.xlsx"
curl -s -X POST localhost:8000/projects/$PID/assess # optional
curl -s -X POST localhost:8000/projects/$PID/recommend # optional
curl -s -X POST localhost:8000/projects/$PID/run
curl -s -o out.zip localhost:8000/projects/$PID/download
Beyond a laptop — a persistent store and a bearer token (see ADR 0025; neither is a substitute for the real Postgres/OIDC roadmap items, both are real and tested today):
IACTRANSLATE_STORE=sqlite IACTRANSLATE_DB_PATH=./iactranslate.db \
IACTRANSLATE_API_KEY=$(openssl rand -hex 24) \
uvicorn iactranslate.api.main:app --port 8000
# every project-touching request now needs:
curl -s localhost:8000/projects/$PID -H "Authorization: Bearer $IACTRANSLATE_API_KEY"
The same IACTRANSLATE_STORE=sqlite switch also persists the audit trail, so
GET /audit still answers after a restart (ADR 0026).
Multi-tenant (shared deployment) — user accounts, login, and per-user project isolation (ADR 0027). Required for any deployment more than one person uses:
IACTRANSLATE_AUTH=session \
IACTRANSLATE_STORE=sqlite IACTRANSLATE_DB_PATH=./iactranslate.db \
IACTRANSLATE_CORS_ORIGINS=https://app.example.com \
uvicorn iactranslate.api.main:app --port 8000
# Register (sets an httponly session cookie), then use it on every call.
curl -s -c jar.txt -XPOST localhost:8000/auth/register \
-H 'content-type: application/json' \
-d '{"email":"you@example.com","password":"a-long-passphrase"}'
curl -s -b jar.txt localhost:8000/projects # only *your* projects
Three things to get right:
- IACTRANSLATE_CORS_ORIGINS must name real origins, not *. Browsers
refuse to send credentials to a wildcard origin, so the web UI silently fails
to authenticate if you use *.
- Serve over HTTPS. The session cookie carries the Secure flag by default;
IACTRANSLATE_COOKIE_SECURE=0 disables that and is for local http testing only.
- IACTRANSLATE_STORE=sqlite is required — accounts and sessions have
nowhere to live in the in-memory store.
Projects created before multi-tenancy was enabled have no owner and are visible only in single-tenant mode; the database migrates itself additively on open, so nothing is lost.
Monitoring — GET /metrics serves Prometheus exposition format. It is
unauthenticated by design (scrapers don't send bearer tokens; the payload is
aggregate counts only, no project or inventory data):
curl -s localhost:8000/metrics
Counters (iactranslate_projects_created_total, …_jobs_failed_total, …) are
process-local and reset on restart — correct Prometheus semantics, since rate()
handles counter resets. A minimal scrape config:
scrape_configs:
- job_name: iactranslate
static_configs:
- targets: ['localhost:8000']
# terminal 1 (API with CORS)
IACTRANSLATE_CORS_ORIGINS=http://localhost:3000 uvicorn iactranslate.api.main:app
# terminal 2 (frontend)
cd web && npm install && npm run dev # http://localhost:3000
Wizard: create project (target + source picker) → upload → optional compare → generate →
download. A one-click sample inventory is provided. Point at another API with
NEXT_PUBLIC_API_URL.
docker build -t iactranslate .
docker run -p 8000:8000 iactranslate # non-root, healthchecked on /health
The Anthropic provider (Claude-powered classification + instance sizing) is reachable from every surface, per-invocation — not just via environment variable (see ADR 0021):
export ANTHROPIC_API_KEY=sk-ant-...
export IACTRANSLATE_ANTHROPIC_MODEL=claude-opus-4-8 # optional
# CLI: explicit per-run, overrides the env-configured default
iactranslate translate rvtools.xlsx --target aws --out ./out --provider anthropic
# API: per-request, so different callers can choose independently
curl -X POST http://localhost:8000/projects \
-H 'Content-Type: application/json' \
-d '{"name":"acme","target":"aws","provider":"anthropic"}'
# Web UI: "Use AI (Claude) for classification & sizing" toggle in step 1
Without a key it silently falls back to rule — but never silently: the CLI prints which
engine actually ran (AI: rule engine (deterministic) [requested 'anthropic' but fell back —
check ANTHROPIC_API_KEY]), the API's run result carries both provider_requested and
provider_used, and the web UI shows an amber fallback banner. MigrationPlan.provider_used
is the one place this is recorded — read it if you need to confirm what actually ran.
The validation layer + catalog guardrail run regardless, so a bad LLM answer degrades
gracefully either way.
The executive report's "Summary" section is also AI-written when (and only when) the plan
itself was AI-assisted (plan.provider_used == "anthropic") — otherwise it's a deterministic
paragraph built from the same facts. The report always shows which mode produced it; see
narrative.py and ADR 0021. This narrative is presentation-layer prose only — it cannot
change the plan or any rendered IaC.
src/iactranslate/targets/oci/ with:
- catalog.py — INSTANCE_CATALOG: List[InstanceSpec] (name, vcpu, memory_gib, family, $/hr) + index().
- mapping.py — VPC_CIDR, FAMILY_BY_TIER, SG_BY_TIER, SUBNET_BY_TIER, DEFAULT_INGRESS, image_key(os).
- __init__.py — an OciTarget class implementing the Target protocol (see aws/__init__.py).
- templates/*.j2 — 11 files (copy aws/templates/, swap resources; use TEMPLATE_MAP).targets/__init__.py _REGISTRY.tests/test_gcp_target.py; the CLI/API/UI pick it up automatically
(they read list_targets()).src/iactranslate/sources/nutanix/__init__.py — a class with name, label,
source_platform, detect(path)→float, parse(path, column_map)→List[RawRecord]. Emit the
raw-record contract (§4). Reuse .._columns.find_column.sources/__init__.py _REGISTRY. Detection uses the confidence scores;
generic stays the floor.scripts/make_fixtures.py and tests in tests/test_sources.py.No pipeline, normalize, validation, generator, CLI, API, or UI changes are needed for either.
All env vars (see src/iactranslate/config.py):
| Env var | Default | Purpose |
|---|---|---|
IACTRANSLATE_LLM_PROVIDER |
rule |
Default engine when --provider/provider isn't given: rule or anthropic. |
ANTHROPIC_API_KEY |
— | Required for anthropic; absent → auto-fallback to rule (see §6.7, ADR 0021). |
IACTRANSLATE_ANTHROPIC_MODEL |
claude-opus-4-8 |
Model for classify/rightsize. |
IACTRANSLATE_MAX_UPLOAD_MB |
25 |
Upload cap → 413. Streamed, never buffered whole. |
IACTRANSLATE_MAX_VMS |
5000 |
Inventory size cap → 400. |
IACTRANSLATE_MAX_PROJECTS |
200 |
Store capacity cap; oldest evicted (temp dirs deleted). |
IACTRANSLATE_STORE |
memory |
memory (dies on restart, zero setup) or sqlite (persists project metadata and the audit trail to IACTRANSLATE_DB_PATH, surviving a restart — see ADR 0025, 0026). |
IACTRANSLATE_DB_PATH |
./iactranslate.db |
SQLite file path when IACTRANSLATE_STORE=sqlite. |
IACTRANSLATE_API_KEY |
(none) | Set to require Authorization: Bearer <key> on every project-touching endpoint. Unset = no auth (today's default). Not OIDC/SSO — see ADR 0025. |
IACTRANSLATE_AUTH |
none |
session enables multi-tenant user accounts, login, and per-user project isolation (ADR 0027). Requires IACTRANSLATE_STORE=sqlite. |
IACTRANSLATE_COOKIE_SECURE |
1 |
Set 0 only for local http testing — it drops the Secure flag from the session cookie. |
IACTRANSLATE_RATE_AUTH |
10 |
Login/register attempts per minute, per IP and per email. 0 disables. Read live — no restart needed. |
IACTRANSLATE_RATE_WRITE |
60 |
Write requests/min per IP (upload, run, jobs, report). 0 disables. |
IACTRANSLATE_RATE_READ |
240 |
Read requests/min per IP. 0 disables. |
IACTRANSLATE_TRUST_PROXY |
0 |
Set 1 only behind a proxy you control, to read the client IP from X-Forwarded-For. Any client can forge that header, so trusting it without a proxy lets attackers bypass every limit. |
IACTRANSLATE_WORKSPACE_ROOT |
(system temp) | Directory for project workspaces (uploads + generated output). Point it at a mounted volume so artifacts survive a container recycle; /tmp does not. |
IACTRANSLATE_APP_URL |
http://localhost:3000 |
Public origin of the web app, used to build the password-reset link. The API's own origin is usually wrong here — the reset form lives in the frontend. |
IACTRANSLATE_SPLIT_COMPUTE_ABOVE |
50 |
Workload count above which compute output is split into compute-<env>-<tier>.tf files for reviewability. 0 keeps a single compute.tf. Purely organizational — no state impact. |
IACTRANSLATE_TARGET_UTILIZATION |
0.65 |
When a source carries utilization, size instances so they run at ~this utilization (right-sizing). |
IACTRANSLATE_PRICING |
static |
static (curated catalog rates, offline) or live (real market prices, cached, falls back to static). |
IACTRANSLATE_GCP_BILLING_API_KEY |
(none) | API key for GCP live pricing (Cloud Billing Catalog). Without it, GCP live falls back to static. |
IACTRANSLATE_PRICE_CACHE |
temp file | Path for the on-disk live-price cache (24h TTL). |
IACTRANSLATE_CORS_ORIGINS |
(none) | Comma-separated allowed origins for the frontend. * = all (dev only). |
NEXT_PUBLIC_API_URL |
http://localhost:8000 |
Frontend → API base URL (web/). |
IACTRANSLATE_E2E_TOFU |
— | Set 1 to run the real tofu validate E2E test. |
TF_PLUGIN_CACHE_DIR |
— | Provider-plugin cache for the tofu E2E (speeds re-runs). |
| Method | Path | Body / notes |
|---|---|---|
GET |
/health |
Liveness. {"status":"ok"}. |
POST |
/projects |
{name, target, source?, column_map?, region?} → 201 project summary. |
POST |
/projects/{id}/upload |
multipart file (.xlsx/.csv). 413 if too big, 400 if wrong type. |
POST |
/projects/{id}/run |
Runs the pipeline synchronously. 200 summary; 422 validation/policy; 400 bad input. |
POST |
/projects/{id}/jobs |
Runs asynchronously; 202 + job_id. Poll /jobs/{id}. |
GET |
/jobs/{job_id} |
Job status (queued/running/completed/failed) + project summary when done. |
GET |
/audit |
Recent audit events (newest first); ?project_id= to scope. Persists across restarts under IACTRANSLATE_STORE=sqlite. |
GET |
/metrics |
Prometheus exposition (counters + in-flight gauge). Unauthenticated — aggregate counts only. |
POST |
/projects/{id}/assess |
Pre-migration readiness assessment (findings + score) from the uploaded inventory. |
POST |
/projects/{id}/recommend |
Cloud recommendation (with decisiveness, annualized cost, notes). |
POST |
/projects/{id}/report |
Executive report HTML. ?include_recommendation=false to skip the 3-cloud compare. |
GET |
/policies |
Available policy rules (name → description). |
GET |
/targets |
Targets and their capability flags. |
GET |
/projects/{id} |
Status + summary (includes the plan's confidence + policy warnings). |
GET |
/projects/{id}/download |
The Terraform project ZIP. 409 if not generated yet. |
DELETE |
/projects/{id} |
Deletes the project + its temp workspace. 204. |
Error contract: 4xx return {"detail": "..."} (422 validation returns {"detail":{"message","issues"[]}});
unexpected errors return a generic 500 (details are logged server-side, never leaked).
Interactive docs at /docs (Swagger) when the server is running.
pytest # full suite (fast, offline) — ~446 tests
ruff check src tests # lint
cd web && npm run lint && npm run build # frontend
# Opt-in: validate generated Terraform against the REAL providers
IACTRANSLATE_E2E_TOFU=1 pytest tests/test_e2e.py::test_generated_terraform_validates
Test layout: test_parsers/normalize/rightsizing/validation/generator (units),
test_targets/azure_target/gcp_target (per-cloud), test_sources (input abstraction),
test_recommend, test_api_security, and test_e2e.py (full source×target API matrix +
real tofu validate).
CI (.github/workflows/ci.yml, 7 jobs, all green): lint, test (3.9/3.11/3.12),
web (npm build), docker (build + container health), terraform-validate (installs
OpenTofu, validates aws/azure/gcp output against real providers).
HEALTHCHECK on /health, runs
uvicorn. Build once, run anywhere.GET /health for probes.rule provider, no customer data leaves the machine.
On anthropic, inventory metadata is sent to the Claude API — use a zero-retention key for
enterprise data. Uploaded files live in per-project temp workspaces and are deleted on
project delete / eviction.POST /projects/{id}/jobs runs the
pipeline on a worker and returns a job_id to poll (GET /jobs/{id}); lifecycle events flow
through an in-process bus; GET /audit returns the trail. These are the interfaces the
production backends drop into — see Deployment & Execution.MAX_VMS cap bounds request cost today; horizontal scale (stateless API pods +
Redis/Celery workers + Postgres + object storage) is the documented v2.1 path.| Symptom | Likely cause | Fix |
|---|---|---|
CLI: unknown target '…' / unknown source '…' |
Typo / unsupported name. | Use aws\|azure\|gcp and auto\|vmware\|hyperv\|kubernetes\|generic\|cloud. |
CLI: No workloads found … |
Source parsed 0 rows — wrong source, or the generic auto-detect missed the name/cpu columns. | Pass --source generic --map "name=…,cpu=…,memory_gib=…" with your real headers. |
API: browser CORS error / network fail |
API started without CORS for the frontend origin. | Start API with IACTRANSLATE_CORS_ORIGINS=http://localhost:3000. |
API: upload returns 413 |
File over IACTRANSLATE_MAX_UPLOAD_MB (25 by default). |
Raise the env var, or trim the export. |
API: upload/recommend returns 400 "could not parse…" |
Corrupt file, wrong extension, or a source forced on a mismatched file (e.g. Hyper-V source on an RVTools xlsx). | Use source: "auto", or match the source to the file. Confirm the file opens in Excel. |
API: run returns 422 with issues[] |
The plan failed validation (bad instance type, CIDR overlap, undefined SG). | Read the issues — usually a source producing odd specs; check the offending VM's cpu/mem. |
run succeeds but terraform apply fails on AMI/image |
Images resolve automatically (AWS aws_ami data sources, Azure source_image_reference, GCP public image families, OCI data "oci_core_images"), but the account/region may lack a match. |
AWS: pin a known-good AMI via ami_overrides in terraform.tfvars. Azure/GCP/OCI: adjust the image reference / data filter in compute.tf. GCP needs a real gcp_project; OCI needs compartment_id + API signing key. |
| Generic source picks wrong/blank columns | Headers don't match the synonym table. | Provide an explicit column_map / --map. Canonical keys: name, cpu, memory_gib\|memory_mib, disk_gib\|disk_mib, os, network, ip, cluster. |
| Cloud source: vCPU/mem come out as defaults | Instance type not in the AWS/Azure catalogs. | Add the type to the relevant targets/*/catalog.py, or include explicit vCPUs/Memory columns in the export. |
tofu validate fails locally |
tofu/terraform not installed, or provider download blocked. |
brew install opentofu; ensure network for tofu init. Set TF_PLUGIN_CACHE_DIR to reuse providers. |
git push rejected: "…without workflow scope" |
Pushing .github/workflows/* with a token lacking workflow scope. |
gh auth refresh -h github.com -s workflow then push (full path /opt/homebrew/bin/gh if not on PATH), or edit the file via GitHub's web UI. |
| Anthropic provider seems ignored | No ANTHROPIC_API_KEY, or SDK not installed. |
It auto-falls back to rule — check provider_used in the run result (CLI prints it, API/UI show a fallback banner) rather than assuming. Install anthropic (in [dev]) and export the key. |
iactranslate: command not found |
venv not activated / package not installed. | . .venv/bin/activate && pip install -e ".[dev]". |
| Web build/lint fails on a hook dep | A useCallback/useEffect missing a dependency. |
Add the dep to the array (ESLint names it). |
Does it need cloud credentials? No — generation is fully offline. Credentials are only
needed when you run terraform apply on the output.
Does it modify the customer environment? No. It reads exported inventory files only.
Is the AI required? No. The default rule provider is deterministic and needs no key;
the AI is an optional refinement, always re-validated.
Can output be deployed as-is? Essentially yes — OS images resolve automatically
(AWS aws_ami data sources, Azure source_image_reference, GCP public image families, OCI
data "oci_core_images", DigitalOcean public image slugs), so there are no image IDs to
hand-fill. AWS/Azure need only cloud credentials; GCP also needs a real gcp_project; OCI
needs a compartment OCID + API signing key; DigitalOcean needs an API token + an uploaded
SSH key fingerprint (see the generated README in each case). The HCL is provider-valid
(CI proves it with tofu validate). DigitalOcean has no Windows Server image at all — see
its generated README for the caveat if the estate has Windows workloads.
Glossary: RVTools = popular VMware vSphere inventory exporter (.xlsx). CMDB =
Configuration Management Database (ServiceNow/Device42/Lansweeper). Rightsizing = choosing
the cloud instance that fits a VM's vCPU/memory with headroom. Target = destination cloud.
Source = origin inventory format. OpenTofu/tofu = open-source Terraform, used to
validate generated HCL.
Measured on Apple Silicon (M-series), Python 3.9, the default rule provider
(no network, no AI), each inventory size run in an isolated process. "Core" is
parse → normalize → agents → validate → render; "end-to-end" additionally
builds the assessment, confidence scoring, executive report, architecture
diagram, and the .zip.
| Workloads | Parse | Core | End-to-end | Peak memory |
|---|---|---|---|---|
| 500 | ~0.02 s | ~0.05 s | ~0.08 s | ~80 MB |
| 1,000 | ~0.035 s | ~0.08 s | ~0.21 s | ~90 MB |
| 5,000 | ~0.16 s | ~0.31 s | ~0.68 s | ~170 MB |
Notes:
- Scaling is roughly linear in workload count; 5,000 (IACTRANSLATE_MAX_VMS
default) is comfortably sub-second end-to-end.
- The anthropic provider and live pricing add network latency (per-call, with
a 24 h price cache) — those paths are I/O-bound, not CPU-bound.
- Reproduce with scripts/ bench or the snippet in the repo; numbers are
indicative and hardware-dependent, not a benchmark guarantee.
IaCTranslate is designed to be safe to run on untrusted inventory uploads and to require the minimum possible trust from the operator.
applys. Running the
output is a separate, explicit act by the user with their own credentials.IACTRANSLATE_MAX_UPLOAD_MB, 413 over).IACTRANSLATE_MAX_VMS).400, never a 500 or a leaked traceback.Unhandled errors return a generic 500 and are logged server-side with context,
never surfaced to the client.
Fail fast. Never emit invalid Terraform. An invalid plan is stopped at the validation gate (ADR 0006); no files are written.
Every validation error is actionable — it explains:
m9.mega not in the AWS catalog"),Surfaces:
- CLI — non-zero exit code with a readable error: … message on stderr.
- API — 422 with {"detail": {"message", "issues": [...]}} for validation;
400 for bad input; 413 for oversized uploads; a generic logged 500 for the unexpected.
Malformed or attacker-influenced input is treated as a normal 4xx, never a crash.
Keep this doc current: when you add a source or target, update §3, §7, and §12; when you add an env var, update §8; when you add an endpoint, update §9. When you add a section, add it to the Contents and (if user-facing) link it from the README.
Why the system is shaped the way it is. Read this to understand how to think about IaCTranslate; read the Operations Guide for how to run and operate it, and the ADRs for the record of individual decisions.
Contents 1. Design principles 2. The canonical model (the key idea) 3. Pipeline phases 4. Decision engines vs analysis engines 5. The policy engine 6. Target capability flags 7. Request flow (sequence) 8. Current scope — what is and isn't supported 9. Assumptions 10. Why not … (how we differ)
These are the invariants every part of the system is held to. When a change would violate one, that's a signal the change is wrong — not the principle.
Deterministic by default. The same input always produces the same output. No hidden state, no time- or network-dependent results on the default path. This is what makes the output auditable and safe to diff in review.
Source agnostic. Everything before normalize is source-specific;
everything after it is source-independent. The pipeline never asks whether
the estate came from VMware, Hyper-V, a CMDB, or a cloud.
Cloud neutral. The cloud recommendation never favors a provider. Weights are explicit and inspectable; no vendor gets a thumb on the scale.
Validation before generation. An invalid plan never produces IaC. The validation layer is a gate, not a warning.
Extensible via registries. New clouds (targets) and new inventory formats (sources) are added behind a protocol — with no changes to the pipeline.
Offline first. The complete pipeline runs with no internet and no API keys. Live pricing and AI are opt-in enhancements that degrade gracefully to the offline path.
AI is optional. AI may improve a structured decision (grouping, instance
choice) but never bypasses validation and never writes Terraform. Turning AI
off changes quality, never correctness. It's reachable per-invocation from
the CLI, the API, and the web UI (--provider/provider/a toggle, not just
an environment variable), and MigrationPlan.provider_used always records
which engine actually ran — a request for AI that silently fell back to the
rule engine is never reported as AI having run (see
ADR 0021). The executive
report's summary paragraph is AI-written under the same condition and
clearly labeled either way.
The single most important design decision is that every source collapses to one representation, and every renderer consumes one representation. Two narrow waists, and everything in between is written once.
flowchart LR
subgraph Sources
V[VMware] --> N
H[Hyper-V] --> N
C[CMDB / spreadsheet] --> N
A[AWS/Azure fleet] --> N
end
N[["NormalizedVM<br/>(canonical inventory)"]] --> P[classify · rightsize · network]
P --> M[["MigrationPlan<br/>(canonical plan)"]]
subgraph Renderers
M --> TF[Terraform]
M --> PU[Pulumi]
M --> FR[future: Bicep, CDK]
end
NormalizedVM is the canonical inventory unit. Every source maps to it;
everything downstream only understands it. The classifier, right-sizer, network
planner, validator, assessment, and confidence engine never learn whether the
source was VMware, ServiceNow, or an Azure export — they read NormalizedVM.
Adding a source is therefore a self-contained job: parse the format into
NormalizedVMs and register it. Nothing downstream changes.
MigrationPlan is the canonical plan. Every renderer consumes it. Terraform,
Pulumi, and any future renderer (Bicep, CDK) never inspect the original
inventory — they read the validated plan. Adding a renderer is likewise
self-contained: consume MigrationPlan, emit files.
This is why "any source → any cloud, any IaC tool" is a small amount of code rather than an N×M explosion: the pipeline in the middle is written once against the two canonical types.
The Infrastructure Graph. Between the plan and the renderers sits a third
artifact: a renderer-neutral topology IR (graph.build_graph) — typed
nodes (VPC, subnet, security group, instance) and edges (contains,
placed_in, secured_by) derived from the plan and shipped as graph.json.
The architecture diagram, CloudFormation, Bicep, and CDK all render from this
graph. Terraform and Pulumi still render their resources from the plan, but
get subnet placement from the graph too (graph.assign_subnets) — one
placement decision every renderer shares, not six independent ones — see
ADR 0010 and
ADR 0016.
Read left-to-right the pipeline is one line; but internally it is four phases with different responsibilities. Naming them is how enterprise migration platforms are usually described, and it clarifies what may touch what.
Discovery Planning Validation Rendering
───────── ──────── ────────── ─────────
parse classify validate (structure) Terraform
normalize rightsize policy (org rules) Pulumi
assessment* network cost/budget (policy) reports*
recommendation* → MigrationPlan ───────────── diagram*
diff* confidence* (all read-only) package · GitOps
costing*
Starred stages are analysis — they read, never write, the plan (next section).
NormalizedVMs and understands the
estate (assessment, recommendation, diff).MigrationPlan.Two kinds of engine operate around the plan, and keeping them distinct is what keeps the architecture clean:
flowchart TD
subgraph Decision["Decision engines — produce the plan"]
C[classify] --> RS[rightsize] --> NET[network] --> REC[recommendation]
end
REC --> PLAN[["MigrationPlan (immutable)"]]
subgraph Analysis["Analysis engines — read-only"]
AS[assessment]
CF[confidence]
DF[diff]
CO[costing]
ER[executive report]
DG[diagram]
end
PLAN --> AS
PLAN --> CF
PLAN --> DF
PLAN --> CO
PLAN --> ER
PLAN --> DG
Decision engines build the plan. Analysis engines (assessment, confidence, diff, costing, executive report, diagram) only read it — they never mutate it. This is the enforced counterpart of design principle #1: because nothing after planning can change the plan, rendering is deterministic and every report describes exactly what gets deployed.
The immutable-plan contract. Once build_migration_plan returns and
validation passes, the MigrationPlan is treated as frozen. Assessment,
confidence, the policy engine, renderers, reports, and GitOps all take it as
read-only input. (See ADR 0007.)
Explainability. Every compute decision carries a human-readable reason
captured at the moment it's made (e.g. "right-sized from observed utilization:
16 vCPU / 64 GiB at 15% CPU → t3.xlarge; database tier"). The package's
decisions.json joins each decision's reason with its confidence — so a
reviewer gets both why an instance was chosen and how sure the tool is,
per workload.
Enterprise requirements diverge exactly at policy: naming conventions, approved instance families, "no public IPs", budget caps, mandatory NAT. Encoding those in the core pipeline would make it an organization-specific fork. Instead they live in a policy engine — a set of pluggable, read-only rules a customer activates and parameterizes through configuration.
flowchart LR
P[["MigrationPlan (valid)"]] --> POL{Policy engine}
CFG[policy config] --> POL
POL -->|deny| STOP[abort — nothing rendered]
POL -->|warn| REP[policy-report.json + render]
POL -->|clean| REN[render]
deny violations abort the run before rendering; warn violations are
reported (policy-report.json) but don't block. Any policy's severity is
overridable per config.no_public_subnets, allowed_instance_families, max_vcpu,
max_monthly_cost, naming_prefix, require_nat. Adding one is a small
registered function — no pipeline change. (See ADR 0008.)// example policy config
{ "no_public_subnets": {}, "allowed_instance_families": {"families": ["t3","m5"]},
"max_monthly_cost": {"budget_usd": 5000}, "naming_prefix": {"prefix": "acme_", "severity": "warn"} }
A Target advertises what it supports as a set of capability flags
(terraform, pulumi, gitops, live_pricing, brownfield_import) rather
than callers branching on target.name == "aws". A UI can enable features
declaratively (GET /targets returns each cloud's capabilities), and adding a
capability to a cloud is a one-line change. (See ADR 0009.)
CLI translate — the deterministic path, no network:
sequenceDiagram
actor User
User->>CLI: iactranslate translate inventory.xlsx --target aws
CLI->>Source: resolve + parse (auto-detect)
Source->>Normalize: raw records
Normalize->>Agents: NormalizedVM[]
Note over Agents: classify → rightsize → network<br/>(rule engine or Claude)
Agents->>Validation: MigrationPlan
Validation-->>CLI: reject if invalid (no IaC emitted)
Validation->>Renderer: valid plan
Renderer->>Packager: {filename: content}
Note over Packager: + assessment · confidence ·<br/>executive report · diagram
Packager-->>User: project dir (+ optional .zip)
API — the same pipeline, one project at a time:
sequenceDiagram
actor Client
Client->>API: POST /projects {name, target, source}
Client->>API: POST /projects/{id}/upload (file)
Client->>API: POST /projects/{id}/assess → readiness + findings
Client->>API: POST /projects/{id}/recommend → ranked clouds
Client->>API: POST /projects/{id}/run → generate
Client->>API: POST /projects/{id}/report → executive HTML
Client->>API: GET /projects/{id}/download → project ZIP
assess, recommend, and report are independent reads over the uploaded
inventory — call them in any order, or not at all. Only run produces the
downloadable project.
Being explicit about the boundary is part of the contract. IaCTranslate migrates server workloads (VMs) and their surrounding network topology — not the platform services layered on top.
Supported
.xlsx, vSphere CSV)kubectl get … -o json — containers sized from resource requests)depends_on chains, rollback strategy, LB-aware downtime estimates —
advisory; does not discover cross-application dependencies it has no
signal for, see ADR 0024)SqliteProjectStore
(IACTRANSLATE_STORE=sqlite), and IACTRANSLATE_API_KEY gates
project-touching endpoints with a bearer token — real, tested stopgaps for
the in-memory-store and zero-auth gaps, not Postgres or OIDC/RBAC
(see ADR 0025)IACTRANSLATE_AUTH=session):
per-user projects, ownership enforced on every project endpoint, cross-tenant
reads and deletes rejected as 404. Not OIDC/SSO, and no orgs/teams
(see ADR 0027)IACTRANSLATE_STORE=sqlite switch) and
Prometheus metrics at GET /metrics — both event-bus subscribers, so routes
and the pipeline carry no instrumentation; counters are process-local by
design, and this is not distributed tracing
(see ADR 0026)Not yet supported
The network model is deliberately VM-centric: VPC/VNet, subnets, tier-scoped security groups, and — where a tier has more than one instance — a load balancer fronting it. Anything above the VM (service meshes, managed data planes, identity) is out of scope by design, not by omission.
The generated plan rests on a small set of stated assumptions. Knowing them is how you read the output critically.
Why not terraform import?
terraform import adopts resources that already exist in the cloud.
IaCTranslate works before migration — it produces the IaC for infrastructure
that doesn't exist yet. (For the brownfield case where a fleet is already in
the cloud, IaCTranslate generates the import blocks for you — see the
Operations Guide.)
Why not Azure Migrate? Azure-only by design; it never emits portable IaC, and a Microsoft tool will never recommend AWS or GCP. IaCTranslate is cloud-neutral, emits Terraform/Pulumi you own, and compares all five clouds unbiased.
Why not AWS Migration Hub / Application Discovery Service? AWS-only; no reusable Terraform; cannot compare clouds. Same structural limits as every first-party vendor tool — the recommendation can only ever point at the vendor's own cloud.
Why not just prompt an LLM to write Terraform? Non-deterministic, unauditable, and unsafe for production infrastructure. Here the LLM (optional) makes only structured decisions that are re-validated; templates emit the actual HCL. Same input → same output, every time.
Two things live here: how a run actually executes today (the execution model, which is real and shipped), and a reference architecture for running IaCTranslate at organization scale (which is a target, not yet built — labeled as such so nobody mistakes the diagram for the current system).
Contents 1. Execution model (shipped) 2. Pipeline stages & trace (shipped) 3. Project state machine (shipped) 4. Single-node deployment (shipped) 5. Reference architecture for scale (not yet built) 6. Multi-tenant model (not yet built)
What happens for one POST /projects/{id}/run (or one CLI translate):
Request
↓ a per-project temporary workspace is created (tempfile dir)
Pipeline (parse → normalize → plan → validate → policy → package [→ zip])
↓ artifacts written into the workspace
Artifacts (Terraform/Pulumi + reports + graph.json + trace)
↓ zipped on download
Response
↓ workspace evicted + deleted when the store exceeds its capacity cap
Cleanup
Bounds that make this safe on untrusted input (all env-overridable — see the Operations Guide §8):
tempfile.mkdtemp), never a shared dir.IACTRANSLATE_MAX_VMS (inventory size cap) and the
25 MB streamed upload cap.IACTRANSLATE_MAX_PROJECTS; oldest projects (and their temp
dirs) are evicted and deleted beyond it.The pipeline runs as an ordered list of named, timed stages; each run emits a
pipeline-trace.json and a structured log line:
| Stage | Does |
|---|---|
parse |
source-specific → raw records |
normalize |
raw records → NormalizedVM[] |
plan |
classify · rightsize · network → MigrationPlan |
validate |
structural gate (catalog, CIDR, refs) |
policy |
organization rules (deny aborts, warn reports) |
package |
render + reports + graph + diagram |
zip |
archive (optional) |
// pipeline-trace.json
{ "total_ms": 38.4, "stages": [
{"stage":"parse","duration_ms":9.0}, {"stage":"normalize","duration_ms":1.1},
{"stage":"plan","duration_ms":1.4}, {"stage":"validate","duration_ms":0.1},
{"stage":"policy","duration_ms":0.02}, {"stage":"package","duration_ms":26.7} ] }
This is the observability substrate. Resumable / distributed execution (persist state after each stage, resume from the failed one, run stages on workers) would build on this stage model — it needs the persistence + queue from the reference architecture and is tracked in the roadmap, not yet implemented.
A project moves through explicit states (visible in the API and the web UI). The
async path (POST /jobs) adds queued → running; the sync path
(POST /run) goes straight to completed/failed:
stateDiagram-v2 [*] --> created created --> uploaded: upload inventory uploaded --> queued: POST /jobs (async) queued --> running: worker picks up running --> completed: success running --> failed: validation / policy / bad input uploaded --> completed: POST /run (sync, success) uploaded --> failed: POST /run (sync, failure) failed --> uploaded: re-upload / retry completed --> [*]: download + delete
The transitions are enforced by the API (Project.status); only a run/job
produces a downloadable project.
The runtime layer is event-driven and job-based today — the interfaces the production backends drop into:
api/events.py) — lifecycle events (project.*, job.*)
published in-process; swap for Redis/Kafka without touching publishers.api/jobs.py) — POST /projects/{id}/jobs → job_id; the
pipeline runs on a worker thread; poll GET /jobs/{id}. Swap the
ThreadPoolExecutor for Celery/Dramatiq + Redis behind the same contract.api/audit.py) — every action recorded via the bus; query
GET /audit. In-memory now; the same subscriber writes Postgres in production.Not yet durable: in-memory jobs/audit don't survive a restart — that comes with the persistent backend below. See ADR 0012.
The shipped deployment is a single container:
Docker (non-root, healthchecked /health)
└─ uvicorn → FastAPI → in-memory ProjectStore + per-project temp workspaces
docker build -t iactranslate .
docker run -p 8000:8000 iactranslate
Good for a team, a demo, or CI. State is in-memory and workspaces are temporary — restart loses projects (by design at this tier).
Not yet built. This is the target topology for multi-user, large-estate operation — what the current single-node design would grow into. It is included so the docs answer "how would this run for 500 users?", not to imply it exists.
flowchart TD LB[Load balancer] --> API1[API pod] & API2[API pod] API1 & API2 --> Q[(Redis / queue)] API1 & API2 --> PG[(Postgres: projects, orgs, policies, audit)] Q --> W1[Worker] & W2[Worker] W1 & W2 --> OBJ[(Object storage: artifacts + ZIPs)] W1 & W2 --> PG
ProjectStore (the store interface is the
seam) and holds orgs/projects/policies/audit.graph.json, Terraform/Pulumi, reports, ZIPs) — reproducible and retained.What already lines up for this: the API is stateless; the ProjectStore is a
single swappable interface; the pipeline is stage-structured with a trace; every
output is a file. The gap is persistence + a queue — deliberately deferred until
there's a multi-user driver (see roadmap).
Not yet built. The data model for SaaS multi-tenancy:
Organization
├─ Users (authn/authz — not yet implemented)
├─ Projects (exists today, in-memory)
├─ Policies (policy engine exists; per-org scoping is the addition)
├─ Artifacts (per-run outputs → object storage)
└─ Audit (per-action log → Postgres)
Authentication, per-org isolation, and audit are the work here; the policy engine and artifact set already exist and would be scoped per organization.
What's shipped, and what's next. "Shipped" means implemented, tested, and green
in CI on main.
Core pipeline
- ✅ Deterministic parse → normalize → agents → validate → render → package
- ✅ Source registry: VMware, Hyper-V, Kubernetes, generic CMDB/spreadsheet, existing cloud fleet
- ✅ Target registry: AWS, Azure, GCP, OCI, DigitalOcean (Terraform) —
OCI's Flex-shape sizing, honest capability sets, and DigitalOcean's real
platform gaps (no subnets, no Windows images) are documented in
ADR 0022 and ADR 0023
- ✅ Rule-engine and Anthropic providers behind one interface (offline default),
reachable from CLI/API/web (not just an env var) with an honest
provider_used record and an AI-written executive-report narrative when
AI actually ran (see ADR 0021)
- ✅ Validation layer (catalog membership, CIDR overlap, referential integrity)
- ✅ Cloud recommender (cost / fit / OS-affinity, unbiased) — and Recommendation 2.0
(decisiveness, annualized cost, estate notes)
- ✅ Utilization-based right-sizing
- ✅ Automatic OS-image resolution (no placeholder AMI/image IDs)
- ✅ Live pricing: Azure (no creds), AWS (boto3), GCP (Cloud Billing Catalog)
Migration-platform layer
- ✅ Assessment engine (readiness score + risk/cost/data-quality findings)
- ✅ Confidence engine (per-decision certainty)
- ✅ Executive report (client-facing HTML)
- ✅ Architecture diagrams (SVG + Mermaid)
- ✅ Infrastructure diff (drift between two snapshots)
- ✅ Brownfield support (Terraform/Pulumi import for existing fleets)
- ✅ Multi-renderer: Pulumi, CloudFormation, Bicep, AWS CDK, Kubernetes/KubeVirt
alongside Terraform — the latter four consume the Infrastructure Graph
directly rather than the plan, proving the IR seam (ADR 0010, 0013-0017)
- ✅ Load balancer topology — multi-instance tiers front behind a load
balancer (ALB / Standard LB / Network LB per cloud), modeled once in
NetworkPlan.load_balancers and rendered by all 6 renderers + the diagram
- ✅ Kubernetes workload discovery — read kubectl get … -o json exports
as a discovery source (containers sized from resource requests) — the input
mirror of the KubeVirt renderer (see ADR 0019)
- ✅ Managed-database re-platforming (advisory) — flags database-tier
workloads as RDS / Cloud SQL / Azure SQL candidates in replatforming.json
without changing the plan (see ADR 0020)
- ✅ Migration wave planning — sequences workloads by tier dependency
(data/cache → app → web) and environment promotion order (dev/test →
staging → production), with depends_on chains, rollback strategy,
validation checks, and LB-aware downtime estimates in waves.json +
the executive report (see ADR 0024)
- ✅ GitOps: opt-in CI/CD workflow (plan on PR, apply on merge)
- ✅ Policy engine — pluggable, read-only org rules (deny/warn) before rendering
- ✅ Capability flags — targets advertise supported features (GET /targets)
- ✅ Explainability — per-decision reason + decisions.json (why + how sure)
- ✅ Infrastructure Graph — renderer-neutral topology IR (graph.json); the diagram renders from it
- ✅ Named, timed pipeline stages — per-stage pipeline-trace.json + structured log (observability)
- ✅ Async jobs + event bus + audit trail (single-node) — POST /jobs → poll → download,
event-sourced lifecycle, GET /audit; the seam Postgres/Redis/Celery/S3 drop into (ADR 0012)
- ✅ Persistent audit + Prometheus metrics — the audit trail survives a
restart under IACTRANSLATE_STORE=sqlite (verified with a live
kill-and-restart), and GET /metrics serves counters + an in-flight gauge in
Prometheus exposition format; both are event-bus subscribers, so the routes
and the pipeline stay uninstrumented (see ADR 0026)
- ✅ Multi-tenancy — user accounts (PBKDF2 passwords), session-cookie login,
and Project.owner_id scoping on every endpoint; cookies work for the
<a href> download/report links that bearer tokens structurally cannot
authenticate (see ADR 0027)
- ✅ Rate limiting + security headers — token buckets per route class, with
auth throttled by source address and by target account (per-IP alone does
nothing against credential stuffing); limits retunable at runtime without a
restart (see ADR 0028)
- ✅ Durable artifacts — IACTRANSLATE_WORKSPACE_ROOT puts generated files
on a real volume instead of /tmp, so a download still works after a restart
(single node; object storage is still the multi-replica answer, see
ADR 0029)
- ✅ Password change + reset — both evict every existing session (a change
that leaves a stolen cookie working is theatre); tokens hashed, single-use,
1h TTL, no account enumeration. Reset delivery is not implemented — the
seam ships a log backend rather than unverified SMTP
(see ADR 0030)
- ✅ Untrusted-input sanitizing — uploaded inventory is sanitized at the
normalize waist, closing a proven template injection (a VM named
${file("/etc/passwd")} was evaluated by Terraform) and fixing Azure, which
produced invalid names for ordinary CMDB data like web server 01
(see ADR 0031)
- ✅ Reviewable output at scale — above 50 workloads, compute is split per
environment/tier (compute-production-web.tf) instead of one 95k-line file.
Purely organizational: Terraform still loads the directory as one config, so
no resource address or state changes
(see ADR 0032)
- ✅ Property-based testing — Hypothesis generates adversarial estates as a
stand-in for customer data we don't have yet. Found four real bugs on the
first run, including distinct machines (web-01 / web.01 / WEB_01)
colliding onto one Terraform label
(see ADR 0033)
- ✅ Zero-friction evaluation — iactranslate demo runs the whole pipeline
on a bundled sample estate (no input file, no account, no upload), plus
docker compose up and a public GHCR image, so a stranger can try it in one
command (see ADR 0034)
- ✅ Realistic RVTools parsing — tested against a 14-sheet, ~50-column
export built from Microsoft's published import spec. Fixed a silent RHEL 8 →
RHEL 9 upgrade, made every OS substitution state itself in the decision
reason, and stopped reading the 11 sheets we never use
(see ADR 0035)
- ✅ Workspace UI — completed steps collapse to summary rows, the result
card comes first, and a project sidebar surfaces work that already persisted
but was unreachable. Page height -42%, step 1 -91%
(see ADR 0036)
- ✅ Inspectable scoring — the recommendation ships its own weights
(cost 0.45 / fit 0.30 / OS 0.25) and names the runner-up, so an architect can
reproduce the ranking by hand. "Unbiased" becomes verifiable rather than
asserted (see ADR 0037)
- ✅ Eligibility gates the recommendation — a cloud that publishes no image
for part of the estate cannot be recommended, however cheap. DigitalOcean was
winning a Windows-heavy estate on cost precisely because it skips Windows
licensing for machines it cannot host
(see ADR 0038)
- ✅ The whole bill, not just the instances — storage, Windows licensing and
load balancers are priced alongside compute, which was 73% of a realistic AWS
estate's cost. Every surface (report, README, main.tf, CLI, API, budget
policy) quotes the same total, and the estimate names what it excludes
(see ADR 0039)
- ✅ Model schema versioning (NormalizedVM / MigrationPlan)
Surfaces & delivery
- ✅ CLI, FastAPI, Next.js web UI
- ✅ Docker image (non-root, healthchecked)
- ✅ CI: lint, pytest (3.9/3.11/3.12), Docker, web build, real tofu validate
Enterprise-platform maturity (the runtime seams — events, jobs, audit, stages,
capability flags, the ProjectStore interface — now exist; these are the durable
backends and integrations that plug into them, via the
reference architecture):
| Milestone | Focus | Builds on |
|---|---|---|
| v2.1 | PostgreSQL store + object-storage artifact store + durable job queue (Redis/Celery) — ✅ metadata persistence and ✅ single-node durable artifacts (IACTRANSLATE_WORKSPACE_ROOT, ADR 0029) shipped now via an opt-in SqliteProjectStore (IACTRANSLATE_STORE=sqlite), a real stdlib-only stepping stone verified with an actual kill-and-restart, not just Postgres itself (see ADR 0025) |
in-memory store, JobQueue, event bus (shipped seams) |
| v2.2 | Desktop app (Tauri) over the same core engine | CLI/API (shipped) |
| v2.3 | AuthN (OIDC/SAML) + RBAC + persistent audit — ✅ multi-tenant accounts, session-cookie login, and per-project ownership (ADR 0027), ✅ a restart-surviving audit trail (ADR 0026), and ✅ a single-token API key for machine callers (ADR 0025). Still open: OIDC/SSO, orgs/teams, RBAC roles, email verification + a reset email backend | audit trail (shipped) |
| v2.4 | Notifications (Slack/Teams/Email) + metrics (Prometheus/Grafana/OTel) — ✅ Prometheus GET /metrics shipped now (counters + in-flight gauge, stdlib-only); OTel distributed tracing and notifications still open (see ADR 0026) |
event bus + pipeline-trace (shipped) |
| v2.5 | CI/CD pipeline generation (Jenkins/GitHub/GitLab/Azure DevOps) | GitOps workflow (shipped for GH Actions) |
| v2.6 | Ticketing (Jira/ServiceNow/Azure DevOps) from the assessment | assessment (shipped) |
| v3.0 | Multi-tenant SaaS + plugin ecosystem + OPA-compatible policy | policy engine, source/target registries (shipped) |
Renderers via the Infrastructure Graph (the IR seam now proven four times):
- ✅ CloudFormation (AWS) — walks graph.json, not the plan
- ✅ Bicep (Azure) — subscription-scope + module, also walks graph.json
- ✅ AWS CDK (Python) — L1 Cfn* constructs, also walks graph.json
- ✅ Terraform/Pulumi placement (subnet assignment) unified onto the graph —
fixed a subnet-collapse bug this surfaced (see ADR 0016)
- ✅ Kubernetes/KubeVirt — VMs as VirtualMachine CRDs, SG ingress as
NetworkPolicy, cloud-agnostic (see ADR 0017)
- ◻ Migrate the rest of Terraform/Pulumi's resource generation onto the graph
- ◻ Rank the recommendation on the full breakdown — recommend() still
compares clouds on compute alone, while every other surface now quotes the
itemized total. Windows licensing varies enough between clouds to move the
ranking, so this is a behaviour change worth making deliberately
- ◻ Live storage pricing — compute already supports --live-pricing; the
storage, licensing and load-balancer rates are hardcoded and dated Aug 2026
- ◻ User-adjustable scoring weights — a cost-insensitive regulated estate
weights fit/OS far higher than 0.45/0.30/0.25; exposing the weights invites it
- ◻ Key identity on VM UUID — real RVTools rows carry a stable UUID; de-dup
and resource labels key on the VM name, so a renamed machine looks new
- ◻ Emit reusable modules with for_each — the per-environment/tier split
(ADR 0032) made output
reviewable; modules would compress ~8k-line files dramatically, but change
what the customer reads, so it is a separate piece of work
Considered, deliberately deferred (tracked so the reasoning is explicit):
src/ reorg into core/ decision/ analysis/ renderers/ — the decision/
analysis separation is real and documented; the physical move is churn with
import-breakage risk that isn't justified yet.These are deliberate boundaries — see Architecture › Current scope.
Priorities are not commitments; this list reflects direction, not a dated plan.
Short records of the why behind the load-bearing decisions in IaCTranslate. Each captures the context at the time, the decision, and its consequences — so the reasoning survives even when the people don't.
Format: Michael Nygard's ADR template.
| # | Decision | Status |
|---|---|---|
| 0001 | A deterministic engine; AI is optional and gated | Accepted |
| 0002 | NormalizedVM as the canonical inventory model |
Accepted |
| 0003 | Clouds behind a target registry | Accepted |
| 0004 | AI behind a provider interface with rule-engine default | Accepted |
| 0005 | Templates (Jinja2) emit IaC, not the model or the AI | Accepted |
| 0006 | Validation is a hard gate before rendering | Accepted |
| 0007 | The MigrationPlan is immutable after planning | Accepted |
| 0008 | A policy engine for organization-specific rules | Accepted |
| 0009 | Targets advertise capability flags | Accepted |
| 0010 | An Infrastructure Graph IR between plan and renderers | Accepted |
| 0011 | The pipeline runs as named, timed stages | Accepted |
| 0012 | Async jobs, an event bus, and an audit trail | Accepted |
| 0013 | CloudFormation renders from the graph; AMIs via SSM | Accepted |
| 0014 | Bicep renders from the graph; subscription-scope + module | Accepted |
| 0015 | AWS CDK renders from the graph via L1 constructs, reusing CloudFormation's AMI logic | Accepted |
| 0016 | Terraform/Pulumi placement moves onto the graph; fixes a subnet-collapse bug | Accepted |
| 0017 | Kubernetes renders VMs as KubeVirt VirtualMachines, not fabricated Deployments | Accepted |
| 0018 | Load balancer topology: modeled once, rendered six ways | Accepted |
| 0019 | Kubernetes as a discovery source: containers read as workloads | Accepted |
| 0020 | Managed-database re-platforming is advisory, not automated | Accepted |
| 0021 | AI made reachable end-to-end (CLI/API/web), and always honestly labeled | Accepted |
| 0022 | OCI target: Flex shapes need a synthetic catalog key, capabilities stay honest | Accepted |
| 0023 | DigitalOcean target: real platform gaps (no subnets, no Windows) stated, not papered over | Accepted |
| 0024 | Migration wave planning: tier + environment order, not a fabricated dependency graph | Accepted |
| 0025 | A real persistent store (SQLite) and real auth (bearer token), scoped to what's buildable without Docker | Accepted |
| 0026 | A persistent audit trail and Prometheus metrics, both built as event-bus subscribers | Accepted |
| 0027 | Multi-tenancy: user accounts, session cookies (not bearer tokens), and per-project ownership | Accepted |
| 0028 | Per-route rate limiting (per-IP and per-account on auth) plus baseline security headers | Accepted |
| 0029 | Generated artifacts move off /tmp onto a durable volume, so files survive with their metadata |
Accepted |
| 0030 | Password change and reset: both evict every session; delivery stops short of unverified SMTP | Accepted |
| 0031 | Untrusted inventory is sanitized once at normalize, killing a proven Terraform template injection |
Accepted |
| 0032 | Large estates split compute output per environment/tier so the result is reviewable | Accepted |
| 0033 | Property-based testing stands in for customer data pre-launch; found 4 real bugs immediately | Accepted |
| 0034 | Zero-friction evaluation: demo needs no inventory, plus compose and a published image |
Accepted |
| 0035 | Test against a structurally real RVTools export; stop silently upgrading a customer's OS | Accepted |
| 0036 | The web UI becomes a project workspace: completed steps collapse, results come first | Accepted |
| 0037 | Publish the recommendation's scoring weights so the ranking is checkable by hand | Accepted |
| 0038 | A cloud that cannot run the estate is not a candidate: truthful image_key, OS-family flags, eligibility gate |
Accepted |
| 0039 | Cost the whole bill — storage, Windows licensing and load balancers, not just instances | Accepted |
See also the Architecture & Design overview.
Status: Accepted
The obvious way to "translate infrastructure with AI" is to prompt an LLM to emit Terraform. That is non-deterministic (the same inventory yields different HCL run to run), unauditable (no explanation for a given resource), and unsafe for production infrastructure, where a hallucinated resource or a silently dropped disk is a real outage. Enterprises evaluating a migration tool ask "can I trust and review the output?" before "how clever is it?".
The core is a deterministic pipeline: the same input always produces the same output. AI, when enabled, makes only structured decisions (which application group, which instance type) that are re-validated; it never writes IaC and never bypasses validation. The default path uses a rule engine and runs with no network and no API keys.
NormalizedVM as the canonical inventory modelStatus: Accepted
Inventory arrives in wildly different shapes: RVTools spreadsheets, Hyper-V exports, arbitrary CMDB columns, cloud fleet CSVs. If every stage of the pipeline (classification, right-sizing, network planning, validation, assessment) had to understand each format, adding a source would mean touching the whole system — an N-sources × M-stages maintenance burden.
Define one canonical inventory type, NormalizedVM (canonical units: vCPU
count, GiB memory/disk), and require every source to map its format to it. The
normalize step is the narrow waist: everything upstream is source-specific,
everything downstream consumes only NormalizedVM.
NormalizedVM → register.
No downstream stage changes. (Symmetrically, 0003
and MigrationPlan do the same on the output side.)NormalizedVM are
dropped (kept only as opaque tags/ids where needed, e.g. external_id for
brownfield). The canonical shape must be expanded deliberately, not casually.Status: Accepted
The product must emit Terraform for AWS, Azure, and GCP — and later, other
clouds — without the pipeline growing cloud-specific branches. Hard-coding
if target == "aws" throughout the classifier, right-sizer, and network planner
would make each new cloud a risky, cross-cutting change.
Model each cloud as a Target behind a protocol: an instance catalog,
tier→family/subnet/security mappings, OS→image resolution, and a set of
Jinja2 templates. Targets live in a registry selected by name. The pipeline
depends only on the Target interface, never on a concrete cloud.
Target protocol is a shared contract: adding a capability
(e.g. image references) means implementing it for every target.Status: Accepted
We want the option of higher-quality decisions from a capable LLM (application grouping, instance selection) without making the product depend on an API key, a network, or a specific vendor — and without letting model output reach production IaC unchecked. Determinism (0001) must remain the default.
Put the classify/right-size steps behind an LLMProvider interface with two
implementations: rule (deterministic engine + static catalog, the default, no
key) and anthropic (Claude structured tool-use). Selecting anthropic without
a key transparently falls back to rule. Provider output is always re-checked
by the validation layer and the catalog guardrail.
Status: Accepted
Something has to turn a validated plan into actual .tf (and .py for Pulumi).
The candidates: have the AI emit HCL (rejected — see
0001), build HCL by string-concatenation in
Python (error-prone, hard to review), or use a templating layer.
Render IaC with Jinja2 templates owned by each target, fed a context built
from the MigrationPlan. build_files(plan, target) returns a {filename:
content} map; the packager writes it. The model carries data; templates carry
the provider syntax. A parallel renderer (renderers/) consumes the same plan
to emit Pulumi, proving the plan is renderer-agnostic.
.tf.j2 files, versioned per cloud —
changing an AWS resource is a template edit, not a code change.MigrationPlan drives multiple renderers (Terraform, Pulumi, future
Bicep/CDK) with no changes to the pipeline.imports.tf) needs no special-casing in the pipeline.tofu validate in CI rather than by the type system.Status: Accepted
Decisions upstream — whether from the rule engine or an LLM — can be wrong: an instance type that doesn't exist, overlapping subnet CIDRs, a security group referenced but never defined, duplicate resource names. If any of these reach the renderer, the tool emits Terraform that fails (or worse, applies incorrectly). For an infrastructure tool, emitting invalid IaC is the cardinal sin.
Insert a validation layer between the agents and the renderer that never
trusts upstream output. It checks catalog membership, CIDR overlap/containment,
duplicate names, and referential integrity. assert_valid(plan, target) raises
PlanValidationError (with the specific issues) and no files are written on
failure — "fail fast, never emit invalid Terraform."
422 with an issues[] list on the API, and a non-zero exit
with a readable message on the CLI.Status: Accepted
A growing set of engines surround the plan: assessment, confidence, the policy
engine, multiple renderers, executive reports, diagrams, GitOps. If any of them
could mutate the MigrationPlan, two things break: the output stops being
deterministic (a report or policy could quietly change what gets deployed), and
reasoning about the system becomes a function of execution order.
Treat the MigrationPlan as immutable once build_migration_plan returns and
validation passes. Everything downstream — analysis engines, the policy engine,
renderers, reports, GitOps — takes it as read-only input. Analysis produces
new artifacts (an assessment, a confidence score, a diff) that reference the
plan; it never edits it.
test_policy_does_not_mutate_plan).Status: Accepted
Enterprise requirements diverge exactly at policy: "all resources tagged", "never deploy public IPs", "only approved VM families", "encrypt disks", "prod only in these regions", "stay under this budget". If these were encoded in the core mappings or validation, every customer would need a fork, and the translation engine would stop being generic. But they also can't be ignored — policy is often the gate an enterprise cares about most.
Add a policy engine: a registry of small, pluggable, read-only policies
that a customer activates and parameterizes through a policy config (JSON).
The engine runs after structural validation, evaluates each configured policy
against the (immutable) plan, and returns violations. deny aborts before
rendering; warn is reported (policy-report.json) but doesn't block. Severity
is overridable per policy. Policies never mutate the plan
(ADR 0007).
--policy), API (policy on create, GET /policies),
and the package (policy-report.json).Status: Accepted
Not every target supports every feature: brownfield import blocks exist for AWS
but not yet Azure/GCP; a future target might lack live pricing or a Pulumi
provider. Encoding these as scattered if target.name == "aws" checks in the
packager, renderers, API, and UI is exactly the kind of cloud-specific branching
the target registry was meant to eliminate — and the
UI has no clean way to know which buttons to enable.
Each Target advertises a set of capability flags — terraform, pulumi,
gitops, live_pricing, brownfield_import — as data. Callers query
capabilities instead of branching on the cloud name; the API exposes them at
GET /targets so a UI can enable features declaratively.
Status: Accepted
MigrationPlan is a good planning artifact, but it is organized for planning
(a list of compute + a network block), not for walking a topology. Consumers that
care about structure — the architecture diagram today, and future renderers like
CloudFormation, Bicep, CDK, or Kubernetes — each re-derive the same graph
(what's in which subnet, what's secured by what) from the plan. That is duplicated
logic and couples every such consumer to the plan's shape.
Introduce an Infrastructure Graph (graph.py): a renderer-neutral topology IR
of typed nodes (VPC, subnet, security group, instance) and edges
(contains, placed_in, secured_by), derived deterministically from the plan
by build_graph(plan). It carries no cloud syntax. The architecture diagram now
renders from the graph (its natural consumer), and the graph ships as
graph.json in every package.
build_graph(plan) rather than the plan directly. Terraform/Pulumi now
share the graph's placement decision too — see
0016, which also documents
a real bug that auditing this shared seam caught.graph.json is a reproducible, tool-agnostic description of the target
topology — useful for diffing, external visualization, or policy tooling.MigrationPlan
today (they work and are proven by tofu validate); migrating them onto the
graph is incremental and unforced. This ADR establishes the IR and proves it
with a real consumer (the diagram), not a big-bang renderer rewrite.Status: Accepted
The pipeline was a straight-line function. That works, but two enterprise needs push toward making the stages explicit: observability (where does the time go? which stage failed?) and, later, resumable/distributed execution (persist after each stage, resume from the failure, run stages on workers). A monolithic function offers no seam for either.
Run the pipeline as an ordered list of named stages — parse, normalize,
plan, validate, policy, package, zip — each timed. The run produces a
PipelineTrace (per-stage duration_ms + total), emitted as a structured log
line and a pipeline-trace.json artifact. The stage names are the same ones the
docs and the state machine use.
Status: Accepted
The synchronous POST /run generates in the request thread. That's fine for a
demo but doesn't scale: long jobs tie up request threads, a restart loses
in-flight work, and there's no record of who did what. The roadmap's v2.1
(Postgres + object storage + a background job queue) is the fix — but building
all of that requires infrastructure (Redis, Celery, Postgres, S3) that isn't
warranted, or testable, at the current stage. We still want the shape now, so
the production backends drop in later without an API rewrite.
Add a runtime orchestration layer, at single-node, behind the interfaces the production backends implement:
api/events.py) — an in-process publish/subscribe. Lifecycle
events (project.created/uploaded/deleted, job.queued/started/completed/
failed) are published; a failing subscriber never breaks publishing. This is
the event-driven seam; a broker (Redis/Kafka) replaces it without touching
publishers/subscribers.api/jobs.py) — POST …/jobs returns a job_id
immediately; the pipeline runs on a ThreadPoolExecutor worker; the client
polls GET /jobs/{id}. Celery/Dramatiq workers replace the executor behind the
same contract.api/audit.py) — subscribes to the bus and records every
consequential action. In-memory (bounded ring) now; the same subscriber writes
to Postgres for permanent history in production.The pipeline stays a pure function — events, jobs, and audit live in the API/runtime layer, not the translation engine (keeping the engine free of the "god object" the reviews warned against).
POST /jobs → poll →
download, with an audit log — the enterprise contract, today, on a laptop.Status: Accepted
ADR 0010 introduced the Infrastructure Graph as
"the intended seam for future renderers" but only proved it with one consumer
(the architecture diagram) — everything that actually emits IaC (Terraform,
Pulumi) still renders from MigrationPlan. A second, different kind of
consumer is the real test of whether the seam is load-bearing rather than
speculative.
CloudFormation is also the first renderer with no cross-cloud equivalent
(Azure/GCP have no CloudFormation), and it lacks Terraform's data "aws_ami" —
there is no built-in CloudFormation construct that resolves "the newest AMI
matching a name filter" the way AMI_FILTERS (targets/aws/mapping.py) does
for Terraform/Pulumi.
renderers/cloudformation.py calls
build_graph(plan) and walks its nodes/edges — VPC → subnets → route
tables/NAT, security groups with their enriched ingress attribute, and
instances via placed_in/secured_by edges — to build the template. It
does not read plan.network or plan.compute directly. This required
enriching the graph first (ADR 0010's nodes previously carried only an
ingress count and no image_key/volume sizes — insufficient for a
renderer, sufficient only for the diagram).{{resolve:ssm:/aws/service/...}}), a real, standard CloudFormation
dynamic reference, for the OSes AWS/Canonical publish one for (Amazon
Linux 2, Windows 2016/2019/2022, Ubuntu 22.04). For OSes with no public SSM
alias (RHEL, SLES, CentOS), emit a plain AWS::EC2::Image::Id template
Parameter with no default — the operator supplies an AMI id at deploy
time (--parameter-overrides AmiIdRhel9=ami-...). This is not a
reimplementation of AMI_FILTERS; CloudFormation has no equivalent to a
name-filter lookup, so the two renderers resolve images by genuinely
different mechanisms.build_cloudformation_files raises
RendererNotSupportedError for azure/gcp, mirroring how the Pulumi
renderer signals unsupported targets.MigrationPlan or the diagram.json.dumps, dict insertion order),
validated in tests as structurally valid CloudFormation (every resource has
Type+Properties, every Ref resolves) and checked with cfn-lint.tofu validate,
there is no equivalent offline proof that a given template deploys cleanly
short of an actual aws cloudformation deploy (or cfn-lint, which checks
structure, not live AWS state).Status: Accepted
ADR 0013 proved the Infrastructure Graph seam with one non-Terraform-shaped consumer. A second, independently-built renderer targeting a different cloud is the real test of whether that seam generalizes, or whether CloudFormation just happened to fit.
Bicep is Azure's native IaC DSL (compiles to ARM JSON) and has no cross-cloud
equivalent, same as CloudFormation for AWS. Unlike AWS AMIs, Azure Marketplace
images are already resolved by a static publisher/offer/sku triple —
targets/azure/mapping.py's IMAGE_REFS / image_reference() — which the
Terraform and Pulumi renderers already call. There is no CloudFormation-style
"AMI resolution" problem to solve for Bicep; image_reference() is reused
as-is.
renderers/bicep.py calls
build_graph(plan) and walks VPC → subnets → security groups (their
enriched ingress attribute) → instances via placed_in/secured_by
edges, exactly like the CloudFormation renderer. It does not read
plan.network/plan.compute for topology, only per-instance scalars
(instance_type, tier, environment) already carried on
ComputePlan/on the graph node.main.bicep is
targetScope = 'subscription': it creates the resource group and calls
resources.bicep as a module scoped to that group. Bicep does not allow
subscription- and resource-group-scoped resources in the same file, so a
single flat file (mirroring how Terraform's azurerm_resource_group +
everything else sit in one file) is not idiomatic Bicep — this ADR chose
the real two-file convention over forcing parity with Terraform's shape.Microsoft.Network/networkInterfaces takes
properties.networkSecurityGroup.id directly, simpler than the Pulumi
Azure renderer's classic association resource.adminPassword is @secure()
with no default (empty string only as the Bicep-required placeholder;
deploy fails without --parameters adminPassword=...); adminSshPublicKey
is likewise required for Linux instances. This is stricter than the Pulumi
Azure renderer's config.get_secret fallback default — a genuine
improvement, not a inconsistency to paper over.build_bicep_files raises RendererNotSupportedError for
aws/gcp.bicep/az CLI in this environment, so
unlike Terraform (tofu validate) or CloudFormation (cfn-lint), the
output is not locally compiled/linted. Tests instead assert structural
properties — balanced braces/brackets, one resource block per graph node,
correct image_reference() wiring, no insecure secret defaults. The
generated README tells the operator to run az bicep build --file
main.bicep before deploying.Status: Accepted
CloudFormation (0013) and Bicep (0014) each proved the Infrastructure Graph seam for a template-shaped IaC format. AWS CDK is a different shape again: a real imperative program (Python, in this codebase's case) that synthesizes a CloudFormation template rather than being one. It is also the closest existing renderer to CloudFormation itself — both target the same AWS resource model — which raises a specific design question: does CDK get its own AMI-resolution logic, or does it reuse CloudFormation's?
CDK's aws_ec2 module offers two families of constructs: L2 (ec2.Vpc,
ec2.Instance) which make opinionated infrastructure decisions on the
caller's behalf (auto-created NAT gateways per AZ, default subnet
configurations, an implicit security group unless overridden), and L1
(ec2.CfnVpc, ec2.CfnInstance, …) which are a near 1:1 mirror of the
CloudFormation resource model — the same shape renderers/cloudformation.py
already builds.
Cfn*) constructs. renderers/cdk.py
walks build_graph(plan) exactly like the CloudFormation renderer —
VPC → subnets → security groups (ingress attribute) → instances via
placed_in/secured_by — and emits ec2.CfnVpc, ec2.CfnSubnet,
ec2.CfnSecurityGroup, ec2.CfnInstance, etc. L2 constructs were
deliberately rejected: their opinionated defaults (e.g. ec2.Vpc
auto-creates a NAT gateway per AZ and its own subnet layout) would silently
diverge from what the plan actually specifies, defeating the point of a
renderer that is supposed to express the plan faithfully._ami_dynamic_ref/_ami_parameter_name from renderers/cloudformation.py
are imported, not reimplemented. CfnInstance.image_id accepts the same
SSM dynamic-reference string CloudFormation does, and CfnParameter is the
CDK-native equivalent of a template Parameter — so the two renderers
solve image resolution identically, because at the L1 level they are
solving the identical problem (there is no CDK-specific mechanism to
invent around).app.py (entry point,
instantiates the stack with the target region as the environment) and
stack.py (the Stack subclass with the actual resources) — plus
requirements.txt and cdk.json so the output is cdk deploy-ready
without hand-editing.build_cdk_files raises RendererNotSupportedError for
azure/gcp.build_graph(plan) call feeds all three with no changes to MigrationPlan.aws-cdk-lib/cdk CLI in this
environment, so unlike Terraform (tofu validate), the output is not
locally synthesized. Tests instead compile() the generated Python
(proving it's syntactically valid) and assert structural properties —
construct counts, correct AMI wiring matching the CloudFormation renderer's
own resolution for the same image keys. The generated README tells the
operator to run cdk synth before cdk deploy.Status: Accepted
ADR 0010 deferred migrating Terraform and
Pulumi onto the Infrastructure Graph, since they already rendered correctly
from MigrationPlan and there was no forcing function to change that.
Auditing the two placement algorithms that existed side by side — the one
build_graph() used to draw placed_in edges (for the diagram, and later
CloudFormation/Bicep/CDK) and the one generator/renderer.py::_assign_subnets
used for Terraform/Pulumi — found they did not agree, and one of them was
wrong:
generator/renderer.py::_assign_subnets (Terraform, and Pulumi via reuse)
correctly round-robins instances of a tier across every subnet of that tier
— spreading instances across availability zones.graph.py's first_subnet_of_tier used dict.setdefault, which only ever
records the first subnet seen for each tier. Every instance of a tier was
placed on that one subnet regardless of how many subnets of that tier
existed. Every renderer that consumes the graph — the diagram,
CloudFormation, Bicep, and CDK — was concentrating all public instances on
one public subnet and all private instances on one private subnet, with
no AZ spread, while Terraform and Pulumi output for the identical plan
correctly spread them.This was not a hypothetical: tests/test_graph.py never asserted more than
"every instance has a placed_in edge," so the collapse-to-one-subnet
behavior shipped in three renderers (0013, 0014, 0015) and the diagram without
being caught.
graph.assign_subnets(plan) is
now the single function that decides which subnet an instance lands in
(the round-robin-across-AZs logic, moved from generator/renderer.py
verbatim — it was already the correct algorithm, just not where every
renderer could reach it). build_graph() calls it to draw placed_in
edges.generator/renderer.py::_assign_subnets (same name, same
signature, so renderers/pulumi.py's existing import needed no change)
now calls build_graph(plan) and reads back placed_in edges, instead of
re-deriving the mapping from plan.network.subnets directly. Likewise
sg_resource (security-group name → resource name) is now read from the
graph's security-group nodes via a new _sg_resource_map, rather than a
separate direct lookup against plan.network.security_groups.external_id, image resolution) straight from
plan.compute/ComputePlan — those aren't topology and don't belong on
the graph. Only the placement relationship (which subnet, which security
group) now flows through the graph.tofu validate (which needs registry
network access this environment doesn't have) that nothing broke.MigrationPlan, not by
walking graph nodes the way CloudFormation/Bicep/CDK do. What moved onto the
graph is specifically the placement decision, which is the part that had
drifted into two disagreeing implementations. Migrating the rest of
Terraform/Pulumi's resource generation onto the graph remains future work,
now with no known correctness gap forcing it._assign_subnets output
is asserted equal to graph.assign_subnets, so the two can never silently
diverge again.Status: Accepted
CloudFormation, Bicep, and AWS CDK (0013– 0015) each proved the Infrastructure Graph seam for a format that mirrors a cloud's own resource model — a VPC is a VPC, an instance is an instance, regardless of syntax. Kubernetes is a genuinely different resource model: Pods, Deployments, Services — nothing in it is a 1:1 analog of a VM, a subnet, or a security group.
The naive move — emit a Deployment per instance — requires inventing facts
the pipeline does not have: a container image, an entrypoint, a listening
port beyond whatever a security group's ingress rules imply, a health check.
ComputePlan/the graph's instance nodes describe a VM (OS image, root/extra
volumes, an instance-type-style vCPU/memory spec) because the source
inventory (RVTools, Hyper-V exports, CMDB rows) describes VMs. Fabricating
containerization details to force a Deployment shape would silently invent
data the plan never asserted.
kubevirt.io/v1: VirtualMachine objects, not Deployments.
KubeVirt is a real, widely-deployed Kubernetes
CRD/operator that runs an actual VM as a cluster-managed workload — the
honest translation of "migrate this VM to run on Kubernetes" rather than a
fabricated one. spec.template.spec.domain.cpu.cores /
resources.requests.memory come straight from the graph instance node's
vcpu/memory_gib; root and extra volumes become one dataVolumeTemplate
PVC each.NetworkPolicy. Each graph security
group's ingress attribute (already enriched per ADR 0010/0013) maps
directly to a networking.k8s.io/v1 NetworkPolicy — CIDR blocks become
ipBlock sources, port ranges become ports entries. This is a clean,
real analog; unlike the VM-vs-container question there was no ambiguity
here.Namespace per project — an explicitly acknowledged approximation.
A Kubernetes Namespace is a naming/RBAC boundary, not a network boundary
the way a VPC is (NetworkPolicy, not the Namespace, is what actually
restricts traffic). Using it as the project's container is the closest
real analog available and is documented as such rather than presented as
equivalent.Service per instance (LoadBalancer for public-subnet-tier
instances exposing a common web port, ClusterIP otherwise), with ports
again derived from the instance's security group — reusing the same
ingress data the NetworkPolicy came from rather than a second, possibly
inconsistent source.target only for
interface parity with the registry and doesn't gate on it.kubectl apply -f x.json works, and a kind: List wrapping multiple
items is standard). This avoids adding PyYAML as a project dependency for
a format json.dumps (stdlib) already produces correctly — the same
choice CloudFormation made (ADR 0013).image_key against. Each
VirtualMachine's dataVolumeTemplates ships with a source: {blank: {}}
placeholder, correctly sized, with the generated README explicitly
instructing the operator to replace it with a real CDI import source
before deploying — the same honesty pattern as CloudFormation's
operator-supplied AMI parameter for OSes with no public SSM alias.build_graph itself.kubectl/KubeVirt to
validate against (tests assert structural validity and referential
integrity instead), and no automatic OS image source (the operator must
supply one via CDI). Both are called out in the generated README, not left
for the operator to discover at deploy time.Status: Accepted
Every renderer modeled a tier's instances as independent VMs with no relationship between them. That's a real gap, not a cosmetic one: a two-instance web tier in AWS/Azure/GCP is not two unfronted public instances in production — it sits behind a load balancer. Modeling that only in the diagram (or only for one cloud) would be half-measures; the point of this change is that the decision ("this tier needs a load balancer") is made once, in one deterministic place, and every renderer + the diagram draws from the same decision.
agents/network.py, not per-renderer.
_plan_load_balancers groups plan.compute by (tier, environment,
subnet_tier); any group with more than one instance gets a
LoadBalancerPlan (models.py) on NetworkPlan.load_balancers. A
single-instance tier gets nothing to front — there is no ambiguity to
resolve here, unlike genuinely close calls elsewhere in the pipeline.LoadBalancerPlan.listeners is built directly from the fronted tier's
SecurityGroup.ingress — the exact ports the tier already declared it
accepts traffic on. Protocol (HTTP/HTTPS) is derived from the port
(443 → HTTPS), the same simple rule everywhere it's used.NodeKind.LOAD_BALANCER and EdgeKind.FRONTS (load balancer → instance)
join the existing PLACED_IN/SECURED_BY edges; a load balancer is
PLACED_IN every subnet of its tier (it spans AZs, unlike a single
instance) and SECURED_BY the same security group its listeners came
from. This is the same seam every graph-consuming renderer already reads.aws_lb/ElasticLoadBalancingV2/
elbv2.CfnLoadBalancer/aws.lb.LoadBalancer depending on renderer).
Azure: Standard Load Balancer (L4) — chosen over Application Gateway
because our listeners are generic TCP/port forwards, not path-based HTTP
routing, and Standard LB is the simpler, more commonly deployed match.
GCP: two genuinely different resource families, not one — an
internet-facing tier gets a classic external Network Load Balancer
(google_compute_target_pool, which takes raw instances directly), while
an internal tier gets a regional internal Load Balancer
(google_compute_region_backend_service, which requires an instance
group even for unmanaged instances). This isn't inconsistency; GCP's own
product boundary runs exactly along that internal/external line.LoadBalancerPlan share a single Service
selecting on their tier/environment labels; only instances with no
load balancer keep the original one-Service-per-VM behavior. This
mirrors the same "front the group" decision every other renderer makes.acm_certificate_arn (template Parameter, CDK
parameter, Terraform variable, or Pulumi config value respectively) and
say so in the generated README/comments, rather than hand-waving a
default value that would fail at deploy time with no explanation.NetworkPlan.load_balancers,
not a separately-derived one.FRONTS/PLACED_IN shape for load balancer
nodes, the diagram's depiction, and — per renderer — that fronted
instances don't also get an individual public IP where one previously
existed.Status: Accepted
Every existing source (vmware, hyperv, generic, cloud) reads a
VM-shaped inventory. But a real estate increasingly includes containerized
workloads on Kubernetes, and "what would it cost / look like to move these off
this cluster" is a legitimate migration question. The input side of the tool
had no way to read them.
Note this is the mirror image of the Kubernetes renderer (ADR 0017, KubeVirt output): 0017 writes VMs into Kubernetes; this reads Kubernetes workloads out as migratable units. Same product, opposite direction.
Two honest problems had to be answered, not hidden:
sources/.kubectl already emits (kubectl get
deployments,statefulsets,pods -A -o json) — no live cluster access, no
Kubernetes client dependency, no credentials. Consistent with the whole
tool's offline, file-in posture. Added is_json/load_json helpers to
sources/base.py (the first non-tabular source).resources.requests, falling back to limits. Summed across
a pod's containers. This is the faithful reading of what the workload
actually declared it needs — not an invented VM allocation. CPU quantities
(500m → 0.5 cores) are ceil'd to whole vCPU (you provision whole vCPUs);
memory quantities (Gi/Mi/M/plain bytes) convert to MiB.volumeClaimTemplates storage becomes the workload's disk.
The one place Kubernetes does declare durable storage; Deployments (which
are typically stateless) get no disks, which is correct.namespace/name) so two same-named
workloads in different namespaces don't collide, and the namespace also
flows through as the cluster field. Tier/environment classification then
works off the name exactly as it does for VMs — no special-casing.spec.os.name / nodeSelector["kubernetes.io/os"] — containers are Linux
unless a Windows node pool is deliberately selected.Source
protocol with zero changes to normalize.py or anything downstream — the
raw-record contract absorbed it, which is the point of that contract..json
file, and the Kubernetes source scores 0.0 for anything that isn't
recognizable Kubernetes JSON, so there's no contention.Status: Accepted
Lift-and-shift puts a database on a plain VM. For a database tier, the cloud-native move is often a managed service (Amazon RDS, Azure SQL, Cloud SQL) that runs backups, patching, HA, and failover for you. Surfacing "these database workloads are candidates for managed services" is real, useful migration guidance.
The temptation is to generate the managed-database IaC. We deliberately do not, and this ADR records why — because the boundary is the whole point.
replatform.py
produces a report (replatforming.json + a migration-summary section)
naming the managed service each cloud offers for the detected engine, with
suggested sizing and caveats — and stops there. A test asserts
analyze_replatforming does not mutate the plan.aws_db_instance with
an empty database would imply an automation we don't perform and can't make
safe — worse than honestly flagging the opportunity.prod-db-01), the report says so and adds a "confirm the
actual database before choosing a managed service" caveat rather than
guessing. Honest unknown beats a confident wrong mapping.Status: Accepted
agents/providers/anthropic_provider.py (Claude-powered classification and
instance sizing) has existed since the project's early scaffolding, selected
via IACTRANSLATE_LLM_PROVIDER=anthropic + ANTHROPIC_API_KEY. But no entry
point ever set it: the CLI had no --provider flag, the API's project-create
body had no field for it, and the web UI had no toggle. The only way to use
it was an undocumented-to-the-user environment variable on the machine
running the process — invisible in the CLI's own --help, unusable per-request
from the API (env vars are process-wide, not per-call), and absent from the
web wizard entirely. Auditing this before adding anything new found the AI
capability was, in practice, dead.
Separately, every report the pipeline produces is deterministic prose driven
by templates (exec_report.py, assessment/) — genuinely useful, but a
"Summary" section written by hand-composed if/else clauses reads noticeably
more mechanical than the surrounding numbers deserve, and is exactly the kind
of short, low-stakes prose an LLM is well suited to improve without
influencing any decision.
MigrationPlan.provider_used is a new field recording which engine
actually classified and sized the plan — "rule" or "anthropic" — set
once, by agents/__init__.py::build_migration_plan, from the resolved
provider's own .name. This is the single source of truth get_provider's
existing silent-fallback behavior needed: requesting anthropic without a
key still returns a working plan (by design), but nothing previously
recorded whether that plan was actually AI-assisted or not.iactranslate translate --provider rule|anthropic resolves and
passes an explicit provider, overriding the environment default for that
invocation. The summary line reports the actual engine used, with an
explicit [requested 'anthropic' but fell back — check ANTHROPIC_API_KEY]
note when they differ — never a silent "AI" claim that isn't true.POST /projects gains a validated provider field ("rule" |
"anthropic", default "rule"), stored per-project and threaded into
run_pipeline. This is the part that actually mattered: an env var can't
let one API caller opt into AI while another doesn't, but a request field
can. The run summary carries both provider_requested and
provider_used, so a client can render the same honest fallback message
the CLI does.AIToggle checkbox in step 1 of the wizard ("Use AI
(Claude) for classification & sizing"), off by default, with inline text
naming the exact requirement (ANTHROPIC_API_KEY on the server) and the
fallback behavior. RunSummary renders an amber fallback banner when
provider_requested !== provider_used, and a green confirmation banner
only when AI genuinely ran — verified live against a running API server
with no key set, producing the fallback banner exactly as designed.narrative.py: a new, small, strictly downstream module that produces
the executive report's "Summary" paragraph. It calls Claude only when
plan.provider_used == "anthropic" (i.e. only when the plan itself was
genuinely AI-assisted — narrative generation piggybacks on that signal
rather than making an independent, possibly-inconsistent choice to call
the API). It is not a decision point: it cannot change the plan, the
render, or any downstream artifact — it reads already-computed facts
(assessment, confidence, cost) and writes prose, nothing else. On any
failure (no key, network, empty response) it falls back to a deterministic
templated paragraph built from the identical facts. The report always
shows a badge naming which mode produced the text — "✨ AI-generated
(Claude)" or "Deterministic summary" — so a paragraph is never mislabeled.get_provider's silent fallback is now
visible everywhere a caller might reasonably ask "did AI actually run?" —
CLI stdout, API JSON, and the web UI — rather than requiring a caller to
infer it from side effects (API latency, absence of an error).agents/base.py intact:
LLMProvider.classify/.rightsize still return typed Pydantic objects the
validation layer re-checks; narrative.generate_narrative returns a string
that only ever lands in a read-only report, never in the plan or any
renderer's input.ANTHROPIC_API_KEY sees identical translate/report output before and
after this change, just now with an honest "Deterministic summary" label
instead of an unlabeled paragraph.Status: Accepted
A fourth cloud target, chosen over DigitalOcean to match the product's enterprise-migration positioning (Oracle shops are a real, distinct migration segment, often paired with Oracle Database workloads). Two design questions this target raised that AWS/Azure/GCP didn't:
VM.Standard.E4.Flex, VM.Standard.E5.Flex) where the actual
OCPU/memory is set independently via shape_config, not a fixed-size SKU
the way t3.xlarge or Standard_D4as_v5 are. The shared
InstanceSpec(name, vcpu, memory_gib, ...) contract needs name to be a
stable, unique catalog key — which a bare shape family string isn't (every
size would collide on the same name).{TERRAFORM, PULUMI, GITOPS, LIVE_PRICING}
capability set; OCI is the first that genuinely can't, honestly.VM.Standard.E4.Flex-2x16
(2 OCPUs, 16 GB). The compute template splits on the first - to recover
the real Terraform shape value; shape_config reads c.vcpu/
c.memory_gib directly — already the catalog spec's values post-rightsizing,
not re-derived from the name. No information is invented; the suffix exists
only because our model needs a unique key and OCI's shape model doesn't
have one built in.capabilities = {CAP_TERRAFORM, CAP_GITOPS} only — test_every_target_advertises_core_capabilities
(previously asserting the full set for every target) was split into two
tests: one asserting AWS/Azure/GCP keep the full mature set, one asserting
OCI explicitly lacks CAP_PULUMI/CAP_LIVE_PRICING. A target with fewer
capabilities than its siblings is a normal, expected state in a capability-flag
system (ADR 0009) — the alternative (claiming capabilities that don't
exist) is the actual bug.data "oci_core_images" (OS + version filter,
sorted by creation time), the same pattern as AWS's data "aws_ami" —
OCI image OCIDs are region-specific, so there's no static portable id the
way GCP's public image families allow. RHEL and Amazon Linux source VMs
map to Oracle Linux (binary-compatible, and OCI's actual default platform
image for that use case) rather than a fabricated substitute.networking.tf says so
explicitly rather than silently limiting availability.tofu validate against the actual oracle/oci provider
schema (not just template rendering) — passed on the first attempt across
all resource types (VCN, subnets, NSGs, instances, load balancers, block
volumes, image data sources) — and via a full browser-driven run through the
web wizard (create → upload → generate → download).recommend() and the CLI/API/web surfaces needed no target-specific code —
list_targets() already drives them dynamically, confirming the target
abstraction (ADR: target registry) scales to a fourth cloud exactly as
designed.capabilities
gains those flags then — not before, and not as an aspirational placeholder.Status: Accepted
The fifth target, and the last item in "New targets" on the roadmap. DigitalOcean's product surface is deliberately simpler than AWS/Azure/GCP/OCI — that simplicity is the product's whole pitch — but it means several things every prior target could assume don't hold here. This ADR is mostly about those gaps, because pretending they don't exist would be the actual mistake.
s-2vcpu-4gb, m-4vcpu-32gb) —
unlike OCI's Flex shapes (ADR 0022), no synthetic catalog key is needed;
InstanceSpec.name is the literal Terraform size value.networking.tf creates
only digitalocean_vpc — nothing else exists to create. The public/private
tier distinction the plan carries is enforced entirely by firewall rules
(security.tf), not by subnet placement, and this is stated in both the
template's own comment and the generated README.digitalocean_droplet resource doesn't expose
that control. Rather than silently under-delivering on "private" tier
semantics, the generated compute.tf and README say so explicitly and
name the real mitigation (firewall-rule scoping, a jump host / VPN).digitalocean_tag (web-fw, db-fw, …); each load
balancer gets its own additional tag applied only to its actual target
instances (not the shared tier tag), so a multi-environment tier (e.g.
web-fw used by both prod-web and dev-web) can't leak an unrelated
environment's Droplet into an LB's backend set.test_digitalocean_windows_source_vms_flagged_in_readme) asserting
the warning actually appears, not just that generation doesn't crash.c.root_volume_gib isn't used for the root
disk (stated in a template comment); extra volumes (storage.tf,
digitalocean_volume) remain genuinely independently sized.capabilities = {CAP_TERRAFORM, CAP_GITOPS} — same honest, narrower
set as OCI (ADR 0022): no Pulumi renderer, no live pricing integration.tofu validate against the actual
digitalocean/digitalocean provider schema (passed first attempt), and a
full browser-driven run through the web wizard (create → upload → generate
→ download) against a live API server — smallest-fit instance selection
correctly fell back from the general-purpose (s-) family to
memory-optimized (m-) where the source VMs' memory requirement exceeded
every general-purpose catalog entry, exercising the existing
smallest_fit fallback path for the first time against a target whose
general-purpose ceiling is lower than the fixture's largest source VMs.recommend(),
the CLI, the API, and the web UI all picked it up automatically via
list_targets(), the same result OCI produced (ADR 0022).Status: Accepted
Enterprises migrate an estate in waves, not one shot — with an order that respects dependencies (a web tier is useless without its app tier) and risk (prove the pattern in a lower environment before touching production). The tool had no notion of execution order at all: every workload rendered as a peer with no sequencing information.
The obvious version of this feature is a full application-dependency graph — "service A calls service B on port 443." This tool cannot build that honestly. It is explicitly offline and file-in only (see architecture.md's scope boundary): no agent, no network flow data, no live discovery. Inventing a dependency edge it cannot observe would be worse than having none — a false edge changes migration order and risk assessment in a way that's actively misleading, not just incomplete.
database/cache/other (0) →
app (1) → web (2). What a layer depends on migrates (and is
validated) first. This is the same Tier enum the classifier already
assigns — no new signal, just an ordering imposed on an existing one.
- Environment promotion order: development/test → staging →
production. Prove the pattern in a lower environment before touching
production — standard practice, and Environment is likewise already on
every ComputePlan.depends_on chains to that same environment's lower-tier-depth wave(s),
transitively (a web wave depends on both its app wave and its data wave).
Different environments never depend on each other — they're independent
estates, safe to run in parallel, and the report says so.WaveReport.notes states the boundary explicitly: cross-application
dependencies (app A calling app B) are real, common, and not modeled —
"if they exist, sequence the affected waves manually rather than trusting
this order blindly." Same honesty pattern replatform.py uses for its own
scope boundary (ADR 0020).lb.targets) gets
an estimate of 0 minutes — a rolling migration through the LB needs no
hard cutover window. An unfronted wave gets a flat, tier-based planning
estimate (30 min for data/cache, 10 min otherwise), explicitly labeled "a
rough planning input, not a guarantee."waves.json is written
alongside every generated project and surfaces in both
documentation/migration-summary.md and the executive report. It never
changes what's rendered — sequencing is planning information, not a
generation input.waves.py for the
explicit boundary).Tier, Environment) that already exist on every
ComputePlan — no new inventory signal, no new classifier logic, and thus
no new failure mode. The wave planner is a deterministic view over data
the plan already carries, in the same spirit as the Infrastructure Graph
(ADR 0010) being a topology view over the same plan.depends_on override
layered on top of the tier/environment default is the natural extension,
not a rewrite.Status: Accepted
A third external architecture review of this project — more accurate than the
two before it — correctly identified two CRITICAL gaps: ProjectStore is an
in-memory dict (a process restart destroys every in-flight project's state),
and the API has zero authentication (any request with network access can
read, run, or delete any project). Both are real. The review's prescribed fix
was PostgreSQL + Redis/Celery and OIDC/SAML + RBAC — the same v2.1/v2.3
roadmap items already on the books.
This environment has no Docker, no Postgres, no Redis, and no identity
provider to build or test against. Building "Postgres support" without a
Postgres instance to run tofu validate-equivalent verification against
would be exactly the kind of unverified claim this project has consistently
avoided (see the honesty pattern in ADRs 0013, 0020, 0023, 0024). The
question this ADR answers: what is the most real progress on these two
CRITICAL gaps buildable and provable with nothing but the Python standard
library?
SqliteProjectStore (api/store.py), selected via
IACTRANSLATE_STORE=sqlite (default remains memory, so nothing changes
for existing users/tests). Project metadata — status, error, summary,
file paths — persists to a local SQLite file (IACTRANSLATE_DB_PATH).
sqlite3 is Python stdlib: no new dependency, no external service.
- Same public interface as the in-memory store (create/get/delete),
plus a save(project) method both implementations expose. The in-memory
store's save() is a documented no-op (its get() already returns the
same mutable object every caller shares); the SQLite store's save() is
load-bearing — every place api/main.py mutates a Project in place
now calls it explicitly, otherwise the mutation would only live in that
request's local variable.
- Verified two ways, not just unit-tested: a pytest asserting a
second SqliteProjectStore instance against the same file sees what
the first wrote (the literal simulation of a restart), and a live
uvicorn process — killed with pkill and restarted — that still
answered GET /projects/{id} with the pre-restart project correctly.
- Honest boundary, stated in the module docstring: this persists
metadata, not the generated files — each project's workspace
(uploads, rendered Terraform) is still a local temp directory. A node
being recycled still loses the files. Durable object storage (S3/GCS)
is the natural next step once a real backend exists to build against —
this ADR doesn't claim to have solved that.api/auth.py), via IACTRANSLATE_API_KEY (unset =
disabled, identical to every prior behavior). When set, every
project-touching endpoint requires Authorization: Bearer <key>, checked
with secrets.compare_digest (timing-safe). /health, /policies,
/targets stay open — read-only capability discovery, not project data.
- Explicitly not presented as OIDC/SSO/RBAC. The module docstring says
so directly: a single shared token is a real, immediately useful
improvement over zero authentication, fully testable without an external
identity provider — and a stopgap, not a substitute, for when real
multi-user identity becomes buildable.sqlite is selected) and access requires a
credential (when one is configured) — provable with tests and a live
kill-and-restart, not just a roadmap line item.create_store()'s env-var-selected-implementation pattern already
generalizes to a third backend, and require_api_key's dependency-based
gating is the same shape a real OIDC token-validation dependency would take.Status: Accepted
Two operability gaps survived ADR 0025.
The first was a self-contradiction in our own code. api/audit.py described
itself as "an append-only record of every consequential action" and noted that
"banks and regulated shops require this" — while being a bounded in-memory
deque that a process restart erases completely. An audit trail that does not
outlive the process is not an audit trail; enlarging the ring does not fix it.
The second: the service emitted no runtime metrics at all. pipeline-trace.json
records per-stage timing within a single run, which is useful for explaining
one translation but tells an operator nothing about the service — how many jobs
failed, how many are in flight, whether upload volume is climbing. External
review flagged observability as a P1, and the v2.4 roadmap line
(Prometheus/Grafana/OTel) had no shipped substrate underneath it.
Both are built as event-bus subscribers, not as instrumentation sprinkled through request handlers. The bus (ADR 0012) already fans out lifecycle events to the audit log; metrics is simply a second subscriber. Adding a metric means handling an event, not editing a route — and the pipeline stays a pure function with no observability concerns compiled into it.
SqliteAuditLog (api/audit.py), selected by the same
IACTRANSLATE_STORE=sqlite switch that selects the project store — one
operator decision, not two — writing to its own table in the
IACTRANSLATE_DB_PATH file. Append-only by construction: the class issues
no UPDATE and no DELETE except a capacity trim that drops only the
oldest rows.
- Verified with a real restart, not just a unit test: a live uvicorn
process created a project, was killed with pkill, and the fresh process
returned the pre-restart event from GET /audit with a byte-identical
timestamp — proof it was read back from disk rather than recreated.
- Honest boundary: SQLite is a real step past "gone on restart"; it is
not a tamper-evident compliance store. That needs Postgres with
append-only grants, or shipping to a SIEM. The module docstring says so.Metrics (api/metrics.py) exposed at GET /metrics in Prometheus
text-exposition format: seven counters (projects created/deleted, uploads,
jobs queued/started/completed/failed) and one jobs_in_flight gauge.
- Stdlib-only, deliberately. The exposition format is a stable line
protocol; emitting it directly avoids taking a prometheus_client
dependency for eight numbers. A test asserts the output parses as valid
exposition (every non-comment line is exactly name value) so the format
claim is enforced, not assumed.
- Unauthenticated, like /health. Prometheus scrapers do not send
bearer tokens, and the payload is aggregate counts only — no project
names, paths, or inventory data. This is a deliberate exception to
ADR 0025's auth gate, justified by the payload carrying nothing sensitive.
- Counters are process-local and reset on restart. This is correct
Prometheus semantics — rate() handles counter resets — and is stated
rather than papered over. Metrics deliberately did not get the SQLite
treatment the audit log did: persisting counters across restarts would
produce worse data by hiding the restart from Prometheus.sqlite selected, the trail genuinely
outlives the process, demonstrated by a kill-and-restart rather than asserted.Status: Accepted
The product direction is a hosted, multi-tenant SaaS. Two things made that impossible, and one of them was a defect in what we had already shipped.
There was no notion of who was calling. ADR 0025
added a single shared bearer token. That is a real improvement over no
authentication at all, but it authenticates a deployment, not a person:
everyone holding the token sees every project. Project had no owner field, so
there was nothing to scope a query by even if we had wanted to.
The bearer token could not secure the product's own UI. The web app exposes
the generated Terraform and the executive report as ordinary links — <a href>
navigations the browser performs itself. There is no fetch call on those, so
there is nowhere to attach an Authorization header. Turning on
IACTRANSLATE_API_KEY therefore 401'd the entire web UI. This was not a missing
line of code; it is a structural property of bearer tokens, and it means the
scheme in ADR 0025 could never have covered the whole product.
Session cookies, not bearer tokens. A cookie is attached by the browser to
navigations as well as fetch calls, which is exactly the property the download
and report links need. Cookies are httponly (XSS cannot read them),
samesite=lax (CSRF-resistant, while still allowing the top-level navigations
that make the links work), and secure unless explicitly disabled for local
http testing.
api/accounts.py). Passwords are PBKDF2-HMAC-SHA256 with a
per-user random salt at OWASP's recommended 600k iterations — stdlib
hashlib, no new dependency. The stored format is self-describing
(pbkdf2_sha256$<iterations>$<salt>$<hash>) so the cost can be raised later
without invalidating existing passwords. Session tokens are stored
hashed, so a database leak yields no usable session.
- Login failures do not distinguish unknown-email from wrong-password, and
the KDF runs even for unknown emails so response time doesn't leak account
existence either. Duplicate registration returns the same generic error
rather than confirming the address is taken.Project.owner_id). Every project belongs to one user, and
_require_project refuses anything owned by someone else.
- 404, not 403. A 403 would confirm the id exists, letting an attacker
enumerate other tenants' projects. The caller cannot tell "no such
project" from "not yours".IACTRANSLATE_AUTH=session turns
multi-tenancy on; unset (the default) leaves the CLI and single-user
self-hosted deployments exactly as they were, with owner_id = None acting
as a single implicit operator. Existing SQLite databases are migrated
additively on open (ALTER TABLE … ADD COLUMN owner_id), so a file written
before this change keeps working and its projects come back as
single-tenant.Writing the boundary tests first found three places where the check was missing entirely. All three are the same mistake — reaching for a resource by id without going through the ownership check — and all three are worth recording because they are the shape of bug this ADR exists to prevent:
DELETE /projects/{id} called store.delete(pid) directly. Any signed-in
user could destroy another tenant's project by guessing its id. This was
destructive, not just a read leak.GET /jobs/{job_id} returned the job's project summary without checking
who owned that project. A job id is a handle to a project and now inherits
its access check.GET /audit returned the whole trail, naming every tenant's project ids
and activity. It is now filtered to projects the caller owns.allow_credentials=True is now required on CORS, which means
IACTRANSLATE_CORS_ORIGINS must name real origins — browsers reject
credentialed requests against a * wildcard. This is a deployment constraint,
not an optional tightening.Status: Accepted
Nothing throttled any endpoint. That was already a denial-of-service problem —
/run does real CPU and disk work per call, and /upload accepts 25 MB — but
ADR 0027 made it sharper by adding
/auth/login: an endpoint that accepts a password and reports whether it was
correct. An unthrottled login is an open invitation to brute-force every account
on the deployment, and it is the single most attackable surface the product has.
Adding authentication without adding a throttle would have been a net negative for security: it creates a credential to guess where none existed before.
Token buckets, applied per route class rather than as one blanket middleware, because the three surfaces have genuinely different economics:
| Surface | Default | Why |
|---|---|---|
/auth/* |
10/min | Accepts a password and reports whether it was right |
| Writes (upload, run, jobs, report, create, delete) | 60/min | Real CPU and disk per call |
| Reads | 240/min | Cheap; only stops runaway clients |
Four decisions worth recording:
X-Forwarded-For is trusted only when IACTRANSLATE_TRUST_PROXY=1. Any
client can send that header. Trusting it unconditionally would let an
attacker bypass every limit by rotating a fake value — the limiter would
look like it worked while enforcing nothing.Limits are read from the environment on every check, not at import. This
started as a testability problem — the suite shares one process and one client
address, so import-time limits could not be varied per test — but it is the
better design regardless: an operator can retune a running deployment, or
disable a limiter with 0, without a restart.
Security headers (X-Content-Type-Options, X-Frame-Options,
Referrer-Policy, Cross-Origin-Opener-Policy) are set on every response.
These matter most on /projects/{id}/report, which returns HTML rendered in the
user's browser. HSTS is sent only over https — asserting it on a plaintext
dev server would pin localhost to https in the developer's browser and break
local work in a way that is annoying to undo.
Retry-After — verified against a live server, not only in tests.tests/conftest.py) because every test
shares one client address and would otherwise look like a single hammering
client; tests/test_ratelimit.py opts back in explicitly, which is where the
limits belong under test.Status: Accepted
ADR 0025 made project metadata
survive a restart, and was explicit that it did not do the same for the
generated files. Every project's workspace came from tempfile.mkdtemp(),
which puts it under the system temp directory.
That is fine for the CLI and local use, and it is wrong for a hosted
deployment. /tmp is periodically cleaned by the OS, is a RAM disk on some
platforms, and is discarded entirely when a container is recycled. The failure
mode is worse than plain data loss: the database keeps a perfectly valid row
pointing at zip_path, so the project still lists as completed and the
download 409s or 500s. ADR 0027 raised
the stakes by putting multiple tenants on one node.
new_workspace() honours IACTRANSLATE_WORKSPACE_ROOT. When set, workspaces
are allocated under that directory — intended to be a mounted volume — instead
of the system temp directory. When unset, behaviour is exactly as before, so
the CLI and existing single-user deployments are untouched.
Both paths go through mkdtemp, which keeps the two properties that matter:
the directory name is unique (no collision between concurrent projects) and it
is created 0700 (one tenant's workspace is not readable by another local
user). Doing this by hand with mkdir would have silently dropped both.
IACTRANSLATE_STORE=sqlite and IACTRANSLATE_WORKSPACE_ROOT set,
metadata and artifacts survive together: verified by generating a project,
killing the server, restarting it, and downloading the identical 22142-byte
ZIP./tmp also survive a plain process restart on the same
host — the real failure modes are container recycling and OS temp cleanup,
neither of which a local restart simulates. The claim here is "files are on a
volume you control", not "we reproduced the outage".rmtree a project's workspace, so the normal lifecycle is handled. But
directories under a durable root now outlive the process, so a wiped database
or an unclean shutdown can leave orphans that nothing will reclaim — worth an
operator's attention in a way a self-cleaning /tmp never was.Status: Accepted
ADR 0027 gave users passwords but no way to change one, and no way back in after forgetting one. For a hosted product that is not a missing nicety — a locked-out customer has no self-service path, and a user who believes their password is compromised has no way to act on it.
The obvious risk in building this is that a reset flow is a second way to authenticate. Done carelessly it is a better attack surface than the login it backs up.
Two flows, split by what can actually be verified here.
POST /auth/change-password — authenticated, requires the current
password. Needs no email, so it is testable end to end and is the primary
path for a user who still has access.POST /auth/forgot-password → POST /auth/reset-password — a
single-use token flow for a user who does not.Both flows call delete_sessions_for_user. Without it a password change is
close to theatre: someone who stole a session cookie stays signed in
indefinitely, and the user who "secured" their account has done nothing to the
attacker. The two flows differ in what happens next, for good reason:
forgot-password returns the same 202 and the same body
whether or not the account exists — verified byte-for-byte. Delivery failures
are swallowed for the same reason: a backend that threw would turn into a 500
that distinguishes real accounts from unknown ones.api/delivery.py defines the delivery seam and ships a backend that logs the
link at WARNING rather than emailing it. That is a deliberate stopping point,
not an oversight.
Writing an SMTP client that has never delivered a message would mean shipping
the one part of this flow nobody has exercised, and the failure would land on a
user locked out of their account, in production. Everything up to the send —
token issue, expiry, single use, enumeration resistance, session eviction — is
tested and verified. set_link_delivery() installs a real sender in one call,
and the natural implementation is a few lines against whatever provider is
already in use.
For a single-operator deployment the logging backend is genuinely usable: the operator reads the link out of the log and passes it on.
Status: Accepted
The product's entire job is turning a file someone uploads into code someone else runs. That makes uploaded inventory untrusted input on a path to code execution, and until now it was passed through verbatim.
Two failures, both found by feeding the pipeline something other than the tidy
fixture names (prod-web-01) that every existing test used.
A VM named x-${file("/etc/passwd")} was written unchanged into a Terraform
tag. Running tofu on the result evaluated the injected function and the
file's contents appeared in the value — confirmed locally, not theorised.
The important detail is that this needed no quote-breaking. HCL evaluates
${...} inside a string literal, so the payload never had to escape its
quotes. An earlier attempt that did try to break out produced only a syntax
error, which is exactly why the interpolation vector is the dangerous one and
the "it just breaks the file" reading would have been wrong.
Impact: a hostile row in a CMDB export — an insider, a compromised discovery
agent, or a client-supplied inventory — yields Terraform that reads local files
into tag values when the consultant runs plan or apply, with whatever
credentials that run carries.
Separately, and arguably worse commercially: names containing spaces,
parentheses, dots, or slashes — web server 01, DB-Prod (Primary),
Exchange/MBX01 — violate Azure's resource-naming rules. Six such names
produced seven tofu validate errors, i.e. every VM in a realistic estate.
AWS, GCP, OCI, and DigitalOcean were unaffected because their templates already
route names through a slug; only Azure used the raw value.
The clean-name fixtures hid both problems completely. This is the more general lesson: the fixtures tested the happy path so thoroughly that the unhappy path was never exercised at all.
Sanitize once, at normalize, rather than in six template languages.
sanitize_identifier() strips characters that are harmless in a hostname but
dangerous in generated code: control characters, quotes, backslashes,
backticks, angle brackets, braces, and $(. It is applied to every
free-text field that reaches a template — name, OS, cluster, network,
datacenter, hostname — not just the name.
The choke point matters. The alternative was correct escaping for HCL, Python
(Pulumi/CDK), JSON (CloudFormation), Bicep, and YAML (Kubernetes) — five
escaping rules, five chances to get it wrong, and a new one every time a
renderer is added. Doing it at NormalizedVM, the narrow waist every renderer
already reads from (ADR 0002), makes
one implementation cover all of them and cover future renderers by default.
Braces are removed outright rather than only the ${/%{ pairs: stripping
just the pairs leaves stray braces that make generated code confusing to read,
and no real hostname contains them.
Azure resource names now use the existing RFC1035 slug, the same helper GCP already used. Tag values keep the original inventory name, so traceability back to the source VM is preserved while the resource name obeys the cloud's rules.
${file("/etc/passwd")} becomes the inert text
x-$-file(-/etc/passwd-). Verified end to end.tofu validate cleanly against both hostile payloads
and messy real-world names — Azure went from 7 errors to 0.prod-web-01,
db.prod.internal, app_server_3, and SRV001 pass through unchanged.unnamed rather than empty,
because downstream de-duplication keys on the name and empty strings would
collapse distinct rows together.tests/test_hostile_input.py keeps both regressions locked, including a
per-cloud check that resource names never contain spaces, parens, slashes,
or hashes.Related finding, not fixed here. A 5,000-VM estate (the documented
MAX_VMS ceiling) runs in ~5s and produces valid Terraform — but it is a
single 95,001-line compute.tf. That validates, and nobody can review it. The
product promise is reviewable IaC, so splitting output per tier or per
application is a real usability gap; it is tracked separately rather than
folded into a security fix.
Status: Accepted
Stress-testing the documented MAX_VMS = 5000 ceiling produced a good result
and a bad one. Good: the pipeline handled 5,000 workloads in ~5 seconds and the
output passed real tofu validate. Bad: it was a single 95,001-line
compute.tf.
That file is valid and unreviewable. The product's central claim is
reviewable, auditable Infrastructure-as-Code — the thing that distinguishes
it from "an LLM wrote some Terraform" — and a file no engineer will ever open
does not honour that claim. This is the failure mode the Terraform community
already names: the monolithic main.tf that nobody wants to touch, where code
review becomes impossible because every change lands in the same giant file.
It is also, notably, the exact weakness of the incumbent in the adjacent space: Terraformer (now archived) was widely described as producing working code with hardcoded values and minimal structure. Output structure is a real axis of product quality, not a cosmetic one.
Above a threshold (IACTRANSLATE_SPLIT_COMPUTE_ABOVE, default 50 workloads),
the compute output is split into one file per environment + tier:
compute-production-web.tf compute-development-web.tf
compute-production-app.tf compute-development-app.tf
compute-production-database.tf …
Three properties made this safe and worth doing:
.tf in a directory
as a single configuration, so splitting changes no resource address, no
dependency, and no state. There is nothing to migrate, and an existing
project re-generated after this change plans identically.Grouping by environment then tier follows the conventional split (by environment, then by component) and matches the two signals the wave planner already sequences migrations on (ADR 0024) — so the files line up with the order the work is actually executed and reviewed in. A reviewer approving "production web" reviews one file.
for_each over a data structure would compress this dramatically and is the
natural next step — it is a much larger change to the templates and to what
the customer reads, so it is deliberately not bundled with this one.main.tf file listing now says compute*.tf, which stays true
whether or not the split applied.0 to restore the previous single-file behaviour.Status: Accepted
The project is pre-customer, so there is no real estate to validate against. Every fixture in the repository was written by the same people who wrote the code, which means the test suite only ever asks questions the authors already thought to ask.
That is not a hypothetical weakness. ADR 0031
documents a proven Terraform template injection and an Azure naming bug that
made output invalid for any realistic estate — both survived 380+ passing
tests, and both were found within minutes of feeding the pipeline input that
did not look like prod-web-01. The fixtures were not wrong; they were
friendly, and friendliness is the blind spot.
"Get a design partner" is the right long-term answer and not something that can be actioned today. The question this ADR answers: what is the best available substitute for a real customer's messy 5,000-row export?
Adopt property-based testing (Hypothesis) for the input path.
Instead of asserting a specific output for a known input, each test states an
invariant that must hold for every possible estate, and the generator hunts
for a counter-example — exploring empty names, zero-CPU rows, Inf cells,
Unicode whitespace, duplicate machines, and control characters that nobody
would think to write into a fixture by hand.
The framing that makes this worth the runtime: an invariant here is a promise to a customer we do not have yet. If one fails, the product is broken for some real estate somewhere, and we would rather find that now than during a pilot.
The invariants asserted are the ones whose violation would be most damaging: parsing never raises, no workload is silently dropped or duplicated, Terraform resource labels are unique and syntactically valid, generated code is never injectable, costs are never negative or NaN, and normalization is deterministic.
Four real bugs on the first run, none of which the existing suite could see:
memory_gib is constrained
> 0, but a row reporting 0 returned 0.0 and raised a ValidationError
out of normalize(). Templates, powered-off shells, and half-filled CMDB
rows all produce zero-memory rows — one of them would have failed a
5,000-VM file. The author had already intended a floor here ("a sane
floor; a VM always has some memory") but it only fired when memory was
missing, not when it was zero.sanitize_identifier matched
\x00-\x1f, but .strip() also removes Unicode whitespace such as U+0085
(NEL) that the class does not cover, so a second pass could shorten the
result again. This matters more than it looks: a name that changes between
runs changes the Terraform resource label, and Terraform treats a renamed
resource as destroy-and-recreate.Inf in a numeric cell raised OverflowError. int(round(float("Inf")))
is not caught by (TypeError, ValueError), so the exception escaped and
failed the upload. pandas produces NaN for blank numeric cells, making this
ordinary input rather than an exotic one.terraform_safe_name
maps every non-alphanumeric run to _, so web-01, web.01, WEB_01, and
web 01 all become web_01 — emitting duplicate resource blocks that
Terraform rejects. A CMDB and a DNS zone rarely agree on separators or case,
so this is close to guaranteed in a real estate. Resolved by suffixing
collisions deterministically at plan-build time, where the whole set is
known, and re-checking so a machine genuinely called web_01_2 is also safe.demo, compose, and a published imageStatus: Accepted
The product was, in practice, un-tryable by anyone outside the repository.
To evaluate it you needed two things: an RVTools or CMDB export of your own, and the willingness to hand that inventory to software you had never run. The second is a large ask. An infrastructure inventory is a map of a company's entire estate — hostnames, IPs, OS versions, capacity. Uploading one to an unknown tool is exactly the kind of thing a security-conscious engineer will not do on a first look, and it is the point at which most evaluations stop.
The irony is that the capability removing this objection was already built and never surfaced. The pipeline is offline by default: no internet, no API keys, no cloud credentials, nothing leaves the machine. That was documented as a security property and never presented as what it actually is — the reason someone can try this without trusting us at all.
Three concrete gaps followed from that framing:
1. iactranslate demo runs the complete pipeline against a sample estate
bundled inside the package. No input file, no account, no upload.
The sample is deliberately realistic rather than tidy: mixed OS (including
Windows Server 2012 R2 and CentOS 7), four tiers, three environments,
utilization data on every row so right-sizing actually engages, multi-disk
machines, and the messy naming real inventories contain — PROD-DB-01,
prod cache 01, dev.sandbox.01. A clean 7-row sample would demo well and
misrepresent what the tool does; this one exercises tier classification,
environment detection, right-sizing, load-balancer topology, and the naming
collision handling from ADR 0033.
The sample ships as package-data, not as a repo file. demo therefore works
from an installed wheel and inside the container, not only from a git checkout
— a test asserts the path resolves relative to the package rather than the
working directory, because that distinction is invisible in local development
and fatal in a published image.
2. docker-compose.yml brings up the stack with persistence already wired
(sqlite store + a durable artifact volume), so down && up keeps projects.
Its memory limit is set from a measurement rather than a guess: a
5,000-workload run — the documented MAX_VMS ceiling — peaks at ~185 MB RSS in
4.5 s, so 512 MB is comfortable.
3. CI publishes to GHCR from main, and smoke-tests the demo path inside
the container. That smoke test is not ceremony: if demo breaks in the image,
nobody can try the product, and every other green check would still pass.
Status: Accepted
VMware is the primary source this product exists to read, and the only RVTools file it had ever parsed was one we wrote ourselves: three sheets, ten columns, tidy names, every field populated.
A real export is a different artifact. Microsoft's
Azure Migrate RVTools import spec
— authoritative because Microsoft had to build a parser against real files —
documents 14+ sheets (vInfo, vHost, vDatastore, vSnapshot,
vPartition, vMemory, vDisk, vCD, vUSB, vNetwork, dvPort, …), a
vInfo sheet of ~50 columns, a VM UUID on every row, and files of up to
20,000 servers.
The gap between those two artifacts is where a first pilot fails. Every other test in the suite could pass and the product could still break on the first real file it was handed.
Generate a structurally realistic fixture (scripts/make_rvtools_fixture.py)
from the documented sheet and column names, at any size, and test against it.
The generator is committed rather than only its output, so the fixture can be
regenerated at 5,000 or 20,000 VMs for performance work without bloating the
repository.
Four real defects surfaced immediately.
The image catalog stocked rhel-9 and no rhel-8, on any of the five clouds,
so image_key returned the newest RHEL available. A customer's certified
RHEL 8 estate would have been provisioned as RHEL 9 with nothing saying so.
rhel-8 is now stocked everywhere, and the mapping honours the reported major
version.
Windows Server 2012 R2 is past end of life and the clouds no longer publish a base image, so a plan genuinely has to fall forward to a supported release. That is defensible. Doing it silently is not: an application certified against 2012 R2 may not run on 2022, and the person reviewing the plan is the only one who can judge that.
Substitutions now state themselves in the decision's reason, which flows into
decisions.json and the executive report. This lives in the planning stage
rather than in assess() deliberately — assessment is target-agnostic by
design (it runs before a cloud is chosen), and the substitution only exists
relative to a specific target's catalog.
The first implementation cried wolf on every Ubuntu machine: RVTools OS strings
end in an architecture suffix, and the 64 in "Ubuntu Linux (64-bit)" read as
a version number. Warnings that fire on healthy machines are worse than no
warnings, because they train the reader to skip them.
sheet_name=None materialised all 14 sheets. On a 5,000-VM file that is ~16,000
rows of vHost/vDatastore/vSnapshot/dvPort data nothing looks at — about
a quarter of parse time, and proportionally more memory at the 20,000-server
ceiling Azure documents. ExcelFile exposes the sheet list without
materialising anything, so the parser now reads only vInfo, vDisk, and
vNetwork, falling back to the first sheet when vInfo is absent (the
non-RVTools workbook path).
Multi-disk aggregation across vDisk rows, powered-off machines being retained
rather than dropped, real RVTools OS strings resolving to images, and detection
of vInfo among 14 sheets — all correct, none previously covered by a test
against a realistic file.
tofu validate
with all 5,000 instances present, split into reviewable per-environment/tier
files (ADR 0032).docker-compose.yml with room to spare.VM UUID is present in real exports and still unused. It is a stabler
identity than the VM name — which is what de-duplication and resource
labelling key on today — and adopting it would make re-runs robust against a
machine being renamed. Left as follow-up rather than folded in here.Status: Accepted
The web UI was five stacked steps in a single scrolling column, and every step stayed fully expanded forever — including the ones you had already finished.
Measured rather than eyeballed, on a 720px viewport:
| Step | Height | Top |
|---|---|---|
| 1 · Create project | 496px | 148 |
| 2 · Upload | 247px | 664 |
| 3 · Assess | 144px | 931 |
| 4 · Compare clouds | 752px | 1095 |
| 5 · Generate | 118px | 1867 |
The page was 2,097px tall. Reaching Generate — the entire point of the product — meant scrolling past ~1,900px of completed work, including a 496px form nobody would touch again. After an action you frequently landed in dead space below the content.
Three smaller problems came out of the same audit:
listProjects() was exported and never called. Projects have persisted
across restarts since ADR 0025/
0029, but the UI had no history and no
resume: every visit began at an empty form and finished work was unreachable.handleViewReport called
bare fetch() without credentials: "include", so the executive report —
the artifact a consultant shows a client — would 401 in multi-tenant mode.Treat this as a workspace over a set of projects, not a wizard run once and abandoned.
acme-datacenter-migration · AWS, rvtools_sample.xlsx,
Readiness 73/100, Recommended: DIGITALOCEAN). They stay re-openable
rather than disappearing, because the point is to compress finished work,
not hide it.listProjects() call that already
existed, with status and headline cost per project. Clicking one resumes it.max-w-3xl (768px) to max-w-6xl — a data-dense tool for
infrastructure engineers was using ~60% of a 1280px window.fetchReportHtml() in the
API client so it inherits the credentialed path every other call uses.overflow-x: auto container.<details> — the API returns reasons for all five.1.00 / 0.64 / 0.60) with no indication that higher is better, and
"Moderate Lead · Margin 0.06" is jargon. Both are information-design problems
in RecommendTable rather than layout, and are better fixed alongside a
decision about what a non-expert reader should take from that table.Status: Accepted
Design principle #3 says the recommendation's "weights are explicit and inspectable; no vendor gets a thumb on the scale." That property is the main reason to trust this ranking over a cloud vendor's own migration tool, which will never recommend a competitor.
The weights were module constants in recommend.py (W_COST = 0.45,
W_FIT = 0.30, W_OS = 0.25) and appeared in no API response. So the claim
was true of the implementation and not of the product: the one question a
cloud architect actually asks — "how are you weighting this?" — could only be
answered by reading the source, which an evaluator will not do.
The audience matters here, and it is not a generalist. This table is read by infrastructure engineers and cloud architects. The right response to "these scores are unclear" is therefore more precision, not simplification — an architect does not want the numbers hidden behind a verdict, they want to check the arithmetic.
Two smaller legibility problems came from the same place:
1.00 / 0.64 / 0.60 with no indication that the
scale is 0–1 or that higher is better. Unreadable for an expert too — this
is missing information, not missing hand-holding.Ship the weights in the response and show them in the table.
Recommendation.weights (ScoringWeights) carries the exact multipliers,
and Recommendation.runner_up names the cloud margin is measured against.Cost ×0.45).margin 0.06 becomes 0.06 ahead of OCI.annual_cost_usd was already in the
payload and unused, and infrastructure budgets are annual.A test asserts every reported weighted_score is reproducible from the
published weights, so the two can never drift: if someone retunes the scoring
and forgets the response, the suite fails rather than the UI quietly lying.
0.45 × 1.00 + 0.30 × 0.64 + 0.25 × 0.60 = 0.79 — which is what makes
"unbiased" a verifiable claim rather than an assertion.capitalize was applied to the
whole decisiveness badge, which title-cased the sentence into "Moderate Lead
· 0.06 Ahead Of OCI". It is now scoped to the single word that needs it.Status: Accepted Date: 2026-08-20
On the realistic 25-VM RVTools estate (8 Windows / 17 Linux), the recommender returned DigitalOcean, scored 0.85 against a runner-up at 0.60, with the reason "Lowest projected cost ($5,976.31/mo)."
DigitalOcean publishes no Windows Server image at all. We knew this — ADR 0023 records it — and the mitigation was a warning in the generated README. That was not enough, because three separate mechanisms combined to bury it:
image_key lied. Every windows-* key in DigitalOcean's catalog maps to
ubuntu-22-04-x64. The plan said windows-2019; the Droplet would boot
Ubuntu. A Windows application server would be provisioned as Linux.
The substitution check was blinded by the lie. ADR 0035's
os_substitution_note() compares the source OS version against the image
key. Source Microsoft Windows Server 2019 against key windows-2019
matched, so the check stayed silent — on the single most consequential
substitution the tool can make. It correctly flagged 2012 R2 → 2022 while
saying nothing about Windows → Linux.
The cost comparison rewarded the gap. DigitalOcean skipped Windows licensing because it cannot run Windows. That is not a discount; it is the absence of the workload. It won on cost by not doing the job.
Presented to a client, this recommends a migration that would fail, and the first cloud architect to read it would stop trusting the whole document.
1. A target's image_key must name the image that will actually be
provisioned. DigitalOcean now returns ubuntu-22.04 for Windows sources.
image_key is a statement of fact about the resulting infrastructure, not a
record of what was asked for. With the lie removed, the existing substitution
check fires on its own.
2. An OS family change is reported differently from a version change. A version substitution warrants "verify application compatibility". A family change means the workload will not boot, and no amount of testing fixes it, so it says so and names the remedy (different cloud, or a custom image).
3. Structured flags, not string prefixes. ComputePlan gains source_os
and os_family_changed. The DigitalOcean README previously detected Windows
workloads with image_key.startswith('windows'); once image_key became
truthful that test found nothing and the warning would have silently vanished —
the fix deleting the warning it exists to serve. Renderers now read the flag.
4. Eligibility gates the recommendation. _unsupported_count() asks each
target's own catalog how many workloads it has no image for. A cloud with any
such workload is marked eligible=False, scores 0.0, cannot be recommended,
and is excluded from the cost baseline that every other cloud's "$X more than
the cheapest" is measured against.
The recommendation for the realistic estate moved from DigitalOcean to OCI, and the quoted spend range from "$71,716 (DIGITALOCEAN) to $192,363" to "$106,966 (OCI) to $192,363" — the earlier low end was never available.
DigitalOcean still appears in the ranked table, with its cost shown "for reference only" and its component scores intact, so a reader can see exactly what it would have scored and why it was excluded. Suppressing it entirely would hide a real option from anyone whose estate is all Linux — where it remains eligible and frequently wins.
weighted_score is no longer weights · components for ineligible clouds.
ADR 0037 published the weights so the ranking could be checked by hand, so the
gate is stated rather than folded into the arithmetic: components stay published,
and the test asserts both the recompute for eligible clouds and the zeroing for
ineligible ones.
Nothing here names DigitalOcean. The count comes from each target's own catalog, so a future target that drops an OS family is caught the same way.
Status: Accepted Date: 2026-08-20
MigrationPlan.total_estimated_monthly_cost_usd is literally
sum(c.estimated_monthly_cost_usd for c in self.compute) — instance cost, and
nothing else. The executive report headlined that figure as "est. cloud spend".
Measured on the realistic 25-VM RVTools estate targeting AWS, the report quoted $16,030/mo. The actual list-price bill for what the plan provisions is $21,866/mo. Compute was 73% of it. Three lines were missing, and the plan already knew all three:
| Line | Missing | Why the plan already knew |
|---|---|---|
| Block storage | $868.80 | root_volume_gib + extra_volumes_gib, 10,860 GiB |
| Windows licensing | $4,835.52 | image_key identifies 8 Windows workloads |
| Load balancers | $131.44 | the plan generates 8 of them |
This is not a rounding problem. It is a 27% understatement, and low is the dangerous direction. A high estimate loses an argument in a meeting. A low one gets written into a budget, overruns six months later, and the consultant who presented it wears that. The tool's entire claim is that its numbers are reproducible from the source inventory; a headline number that omits a fifth of the bill undermines every other figure in the document.
A new costing.py produces a CostBreakdown — compute, storage, Windows
licensing, load balancers, total — from list-price, on-demand rates.
It is an analysis engine. Like assessment, confidence, waves and the diagram, it reads the immutable plan and never mutates it (ADR 0007). A test asserts the plan's serialization is byte-identical before and after.
No committed-use discount is applied. Reserved Instances, Savings Plans and CUDs routinely cut compute 30–60%, and applying them would produce a much more attractive number. Quoting a discount the customer has not actually purchased is how an estimate becomes wrong in the customer's favour, so the report states the discount exists and that it is deliberately excluded.
The estimate states its own boundaries. A "not included" list ships with every breakdown: egress, backup/snapshots, support plans, post-migration managed services, and the migration project itself. None are derivable from an inventory export — nothing in an RVTools file says how much data an application egresses — so the honest move is to name them rather than to let the total imply a quote.
The tier table is relabelled. It sums to compute only. Left titled "Cost breakdown by tier" next to a larger headline, its total reads as a contradiction, so it is now "Compute by tier", its shares are of the compute subtotal, its footer says "Compute subtotal", and a line states the shares are not of the headline total.
The narrative quotes the same total, so the prose and the table cannot disagree.
Every quoted figure rises. On the realistic estate, AWS moves $16,030 → $21,866, Azure $12,498 → $17,633, GCP $11,033 → $16,026, OCI $8,914 → $13,098.
Compute's share varies by cloud (68–84%), so the correction is not a flat multiplier and the relative ranking of clouds can shift — which is the point.
The recommendation engine still ranks on compute cost alone. Bringing the full
breakdown into recommend() would change the ranking and is the natural next
step, deliberately kept out of this change so the cost model can be reviewed on
its own before it moves a recommendation.
Rates are a maintenance burden: five clouds × three rate tables, hardcoded and
dated August 2026. They will drift. That is accepted for now — a wrong-by-drift
number beats a wrong-by-omission one, and every rate carries its source in a
comment. Live pricing already exists for compute (--live-pricing); extending
it to storage is the eventual fix.