terraform_state_reader

A small Go module that reads Terraform and OpenTofu state (format version 4) from a byte stream and emits a normalised, flattened view of it: one record per line as JSONL. It ships as a stdlib-only library (tfstate) and a CLI of the same name.

It exists to be vendored into StackQL and exposed as an MCP tool that surfaces the last known state of infrastructure so an agent can compare it against live StackQL queries. Mapping Terraform resource types to StackQL resources and drift comparison live in the consumer, not here.

Install

go install github.com/stackql/terraform_state_reader/cmd/terraform_state_reader@latest

Static binaries for linux, darwin and windows (amd64 and arm64) are attached to each GitHub release.

Library:

go get github.com/stackql/terraform_state_reader/tfstate

Requires Go 1.23 or later.

CLI usage

terraform_state_reader [flags] [path]

  path           state file; "-" or omitted reads stdin
  --kinds        comma list: meta,resource,output (default all)
  --mode         managed|data (default both)
  --type         glob on resource type, repeatable
  --address      glob on address, repeatable
  --include-sensitive
  --pretty       indented JSON instead of JSONL (debugging only)
  --summary      print counts by type to stderr and exit 0
  --version

Records go to stdout, errors and --summary output go to stderr. Exit codes: 0 success, 1 parse or IO error, 2 usage error.

The CLI only reads local files and stdin. For remote state, pipe it in:

aws s3 cp s3://my-bucket/env/terraform.tfstate - | terraform_state_reader --mode managed
gsutil cat gs://my-bucket/env/default.tfstate | terraform_state_reader --type 'google_compute_*'
terraform_state_reader --summary terraform.tfstate

Globs support * and ? only. Character classes are not supported because [ and ] appear in resource addresses, so --address 'aws_subnet.private[*]' works as written.

Library usage

import "github.com/stackql/terraform_state_reader/tfstate"

st, err := tfstate.Parse(r) // r is any io.Reader the caller opened
if err != nil {
    return err
}
for rec, err := range st.Records(tfstate.Options{Modes: []string{"managed"}}) {
    if err != nil {
        return err
    }
    if rec.Kind == tfstate.KindResource {
        fmt.Println(rec.Resource.Address, rec.Resource.Type)
    }
}

Other entry points:

• tfstate.WriteJSONL(w, st, opts) writes the record stream to w.
• tfstate.Stream(ctx, r, opts, emit) parses and calls emit per record; stops on context cancellation or the first error.
• tfstate.ParseProvider(raw) and tfstate.ParseAddress(s) are exposed for consumers that need to take provider strings or addresses apart.

Errors from Parse wrap one of ErrNotTerraformState, ErrUnsupportedVersion (message includes the version found) or ErrEncryptedState; test with errors.Is.

Sensitive values are redacted to "<sensitive>" unless Options.IncludeSensitive is set. Attribute numbers are decoded as json.Number so large integers round-trip exactly.

Record schema

One JSON object per line, discriminated by kind. Empty optional fields are omitted. Consumers must not rely on field order.

{"kind":"meta","format_version":4,"terraform_version":"1.9.5","serial":42,"lineage":"5a2b...","resource_count":18,"instance_count":23,"output_count":3}
{"kind":"resource","address":"module.vpc.aws_subnet.private[0]","module":"module.vpc","mode":"managed","type":"aws_subnet","name":"private","index_key":0,"provider":{"hostname":"registry.terraform.io","namespace":"hashicorp","name":"aws","alias":"","raw":"provider[\"registry.terraform.io/hashicorp/aws\"]"},"schema_version":1,"attributes":{"id":"subnet-0abc","vpc_id":"vpc-0def","cidr_block":"10.0.1.0/24"},"sensitive_paths":["tags.secret"],"dependencies":["module.vpc.aws_vpc.main"]}
{"kind":"output","name":"vpc_id","value":"vpc-0def","type":"string","sensitive":false}

• meta is always first. Its counts describe the whole state, not the filtered stream.
• address is the full instance address: [module.<path>.][data.]<type>.<name>[index]. Integer keys render as [0], string keys as ["key"] with quotes and backslashes escaped.
• provider is parsed from the raw v4 string. provider["host/ns/name"], the aliased form provider["..."].alias, module scoped module.x.provider["..."] and the legacy provider.aws form are all handled; raw is always preserved. Legacy form leaves hostname and namespace empty.
• status is tainted when the instance is tainted, otherwise omitted.
• attributes is the instance attributes object as stored, keys untouched. Legacy flatmap instances (attributes_flat) are emitted as a flat string map.
• sensitive_paths is derived from the v4 sensitive_attributes field and rendered as dotted paths: result, rules[0].password, tags.secret. Map keys that are not simple identifiers render as tags["a.b"]. The list is emitted whether or not values were redacted.
• Redaction walks each path and replaces the leaf. Dynamically typed attributes stored as {"type":...,"value":...} (for example terraform_data.input) are unwrapped during the walk. If a path cannot be followed, the deepest value reached is redacted rather than leaving a marked value exposed.
• Outputs with sensitive: true have value redacted under the same opt-in. type is the raw type constraint from state.
• The v4 private blob is never emitted. check_results, identity and create_before_destroy are ignored in the record output (CreateBeforeDestroy is available on the parsed State).

Testing

make test        # go test -race ./... (no network, no external binaries)
make lint        # gofmt, go vet, golangci-lint
make fuzz        # short fuzz of the parser
make live        # apply testdata/live with terraform and tofu, verify the reader (needs network)
make fixtures    # regenerate testdata/{terraform,tofu}/*.tfstate and the goldens

testdata/terraform and testdata/tofu hold states written by real binaries (two minor versions of each) from the configuration in testdata/live. testdata/edge holds hand-written cases: v3, OpenTofu encrypted, malformed, empty, legacy providers, flatmap attributes, unusual index keys and sensitive paths. testdata/golden holds the expected JSONL for every parseable fixture, redacted and with --include-sensitive. Regenerate goldens with go test ./tfstate -run TestGolden -update only when the change is intentional.

The live tests (go test -tags live ./live/...) run init, apply and taint with each binary found on PATH using only the null, random and local providers plus the builtin terraform_data, so they need no credentials. CI runs them across a matrix of Terraform and OpenTofu versions. TFSR_LIVE_TOOLS overrides the binaries used and TFSR_LIVE_REQUIRE=1 makes a missing binary a failure instead of a skip.

Not supported

• State format versions other than 4 (Terraform before 0.12). ErrUnsupportedVersion is returned with the version found.
• OpenTofu encrypted state. ErrEncryptedState is returned; decrypt before passing it in.
• Fetching state from remote backends. Pipe the bytes or have the calling application fetch them.
• terraform show -json output. Different format.
• Inferring whether Terraform or OpenTofu wrote the state. terraform_version is emitted as found.
• Renaming or reshaping attribute keys.

Licence

MIT.