Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions .claude/skills/tirith-policies/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
---
name: tirith-policies
description: Write, validate, run and debug Tirith IaC governance policies, install Tirith, and add it to a CI pipeline (GitHub Actions, GitLab CI, Bitbucket Pipelines, Jenkins, Azure DevOps, CircleCI or any container runner). Use when writing or editing files under .tirith/policies, when a Tirith check fails in CI, when asked to add a guardrail to a Terraform or OpenTofu pipeline, or when reading a Tirith result document or exit code.
---

# Tirith

Tirith evaluates the plan a pipeline already produces against declarative JSON policies, and
exits non-zero so a violating change never reaches `apply`.

A policy is **JSON data, not a program**. It names a provider, the value to inspect, and the
condition that value must satisfy. Tirith does the traversal and returns resource-level evidence.

## Install: not from PyPI

`pip install tirith` installs an **unrelated project of the same name**. `pip install py-tirith`
finds nothing: that is the package name in `setup.py`, and it is not published. Install from git,
pinned to a tag:

```bash
pip install "git+https://github.com/StackGuardian/tirith.git@1.2.0"
```

## The one rule

**Never hand back a policy you have not run against a document that should fail it.**

A policy that matches nothing looks identical to one that works: same shape, same silence. Run it
against input you expect to be refused. If that run exits `0`, the policy matched nothing and
gates nothing.

```bash
tirith -policy-path .tirith/policies -input-path plan.json --fail-on-error
```

## Exit codes are a contract

| Exit | Meaning | What CI should do |
| --- | --- | --- |
| `0` | Every check passed | Continue to `apply` |
| `3` | A policy failed | Fail the job: the change was refused |
| `1` | No verdict could be reached | Fail the job, but report a **tool or input** problem |

`ExitStatus.ERROR_TIMEOUT = 2` is declared in `status.py` and returned nowhere, including on the
platform path, which maps a timeout to `1`. Do not branch a pipeline on it.

`3` is deliberately not `1`. Collapsing them reports an outage as a policy violation, and a job
that cannot tell them apart cannot tell a working gate from a broken one.

**`final_result: null` is not a pass.** It means every check was skipped, so the policy evaluated
nothing. It exits `1`.

## Write a policy

Work in this order. Guessing any of the four is the main source of silently-broken policies.

1. **Which document are you reading?** An OpenTofu or Terraform plan, a Kubernetes manifest, an
Infracost breakdown, or arbitrary JSON or YAML. That fixes `meta.required_provider`. There are
five providers and **no CloudFormation provider**: a CloudFormation template is arbitrary JSON,
read by `stackguardian/json`.
2. **Which operation?** Each provider exposes a closed set: see `reference/schema.md`.
3. **Which key names the value?** It differs per provider, and the wrong one is *ignored* rather
than rejected, so the check reads nothing and passes. See `reference/schema.md`.
4. **Which condition?** Thirteen, listed in `reference/schema.md`. There is no `Exists`.

```json
{
"meta": {
"version": "v1",
"required_provider": "stackguardian/terraform_plan",
"name": "Every resource carries a costcenter tag"
},
"evaluators": [{
"id": "costcenter_tag_present",
"description": "Every taggable resource declares a costcenter tag",
"provider_args": {
"operation_type": "attribute",
"terraform_resource_type": "*",
"terraform_resource_attribute": "tags.costcenter"
},
"condition": {"type": "IsNotEmpty"}
}],
"eval_expression": "costcenter_tag_present"
}
```

`eval_expression` combines evaluator **ids** with `&&`, `||`, `!` and parentheses. An evaluator
the expression never names cannot affect the verdict. `!` is the only negation mechanism: there
are no inverse conditions, so write the positive detector and invert it.

## Four traps that cost the most time

**`error_tolerance` goes inside `condition`, not on the evaluator.** On the evaluator it is
silently ignored: no warning, and the check still fails.

```json
{"condition": {"type": "IsNotEmpty", "error_tolerance": 2}}
```

**One evaluator produces one result per matching resource.** A plan with three buckets gives three
results from one rule, and the check fails if any of them fails. That is the mechanism, not a
wildcard trick.

**A missing attribute is severity 2, a missing resource type is severity 1.** With
`error_tolerance: 2` a resource lacking the attribute is *skipped* rather than failed, which can
turn the whole policy into `final_result: null`. Skipping is not passing.

**`tirith lint` is not in the released package.** It is in development. The released CLI dispatches
`tirith`, `tirith ui` and `tirith platform check` and nothing else, so do not put it in a pipeline
you are writing for someone. Check policy shape by reading `reference/schema.md` and by running
the policy. See `reference/validate.md`.

## Before you hand it back

1. Does `eval_expression` reference every evaluator you wrote?
2. Is every `condition.type` in the closed list of thirteen?
3. Is `error_tolerance`, if used, inside `condition`?
4. Did you **run it** against a document that should fail it, and did it exit `3`?

## Reference

| File | Use it for |
| --- | --- |
| `reference/schema.md` | The closed vocabulary: conditions, providers, operations, argument keys |
| `reference/validate.md` | Checking a policy is well-formed, and the traps to check by hand |
| `reference/verdicts.md` | Running a policy, exit codes, and finding the resource behind a failure |
| `reference/terraform-plan.md` | The plan provider's operations, for OpenTofu and Terraform |
| `reference/other-providers.md` | Kubernetes, Infracost and arbitrary JSON or YAML |
| `reference/variables.md` | One policy across environments with `-var` |
| `reference/install.md` | Installing Tirith, and why the install is a git URL |
| `reference/pipelines.md` | GitHub Actions, GitLab CI, Bitbucket, Jenkins, Azure DevOps, CircleCI |
| `reference/platform.md` | Evaluating against an organization's central policies |
| `reference/debug-ci.md` | Starting from a red build and ending at the rule and the resource |

Worked policy/input pairs live in `src/tirith/tui/examples/` in the Tirith repository.
72 changes: 72 additions & 0 deletions .claude/skills/tirith-policies/reference/debug-ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Debug a red CI check

Start from a failed build and end at the rule and the resource. Work in this order — it is
ordered by how often each step is the answer.

## 1. Which exit code?

```bash
echo $?
```

| Exit | What it means | Where to look |
| --- | --- | --- |
| `3` | A policy ran and refused the change | Step 2 — this is a real verdict |
| `1` | No verdict was reached | Step 4 — this is not a violation |
| `0` but you expected a failure | Nothing was in scope, or `--fail-on-error` is missing | Step 5 |

A job that collapses `1` and `3` will send you to step 2 for a problem that lives in step 4. Fix
the job's exit-code handling first if it does that.

## 2. Which check failed, and on which resource?

```bash
tirith --json -policy-path .tirith/policies -input-path plan.json > result.json
```

In `result.json`, find the evaluator with `"passed": false`. Each entry in its `result[]` array
carries a `meta` with the resource: `address`, `type`, and the `change` with `actions`, `before`
and `after`. Name the resource from `meta.address` rather than quoting the message — on a wildcard
policy every message reads identically.

`tirith ui --result result.json` opens the same document in an explorer, if the extra is installed.

## 3. Is the finding correct?

Read `change.after` for the address and compare it against `condition.value`. Three outcomes:

- The value really does violate the rule → fix the Terraform.
- The value is fine but the rule tests the wrong thing → fix the policy.
- The value is **absent** → the failure will arrive *without* a resource address, because there is
no value to attach one to. Search the plan for the resource lacking that attribute.

## 4. Exit `1` — no verdict

Check `final_result` in the result document.

- **`final_result: null`** — every check was skipped, so nothing was evaluated. Almost always
`provider_args` matching nothing. Verify the resource type is in the plan, then the attribute
path, then whether `error_tolerance` is forgiving the very thing you meant to catch.
- **No `final_result` at all** — the policy could not be loaded. Usually an unresolved variable;
read the `errors` array.
- **A misconfigured policy** — an unsupported `condition.type` or unknown provider arrives as an
ordinary failed check and exits `3`, not `1`. Check `condition.type` and every `provider_args`
key against `reference/schema.md`: an unknown key is ignored rather than rejected, so the fault
is in the policy even though the failure points at infrastructure.

## 5. Exit `0` when you expected a failure

- Is `--fail-on-error` present? Without it the exit code is always `0` and the verdict is only in
the output.
- Did the policy match anything? A cost policy over a misspelled `resource_type` sums to `0` and
passes. A wildcard attribute policy with `error_tolerance: 2` skips every resource lacking the
attribute.
- Prove the gate works by running it against a document that **should** fail it. A guardrail only
ever seen passing is a guardrail nobody has tested.

## The two commands

```bash
tirith --json -policy-path .tirith/policies \
-input-path plan.json | head -40 # what did it actually decide?
```
55 changes: 55 additions & 0 deletions .claude/skills/tirith-policies/reference/install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Install Tirith

## Tirith is not on PyPI

`pip install tirith` installs an **unrelated project of the same name**, and `pip install
py-tirith` finds nothing. Always install from git.

```bash
pip install git+https://github.com/StackGuardian/tirith.git
```

Pin a tag rather than tracking the default branch, so a CI job cannot change behaviour underneath
you:

```bash
pip install "git+https://github.com/StackGuardian/tirith.git@1.2.0"
```

`git ls-remote --tags https://github.com/StackGuardian/tirith.git` lists the available tags.

## Verify

```bash
tirith --version
```

## Python versions

| | Needs |
| --- | --- |
| Tirith itself | Python 3.8 or newer |
| The `[tui]` extra (`tirith ui`) | Python 3.9 or newer |

## The optional interface

`tirith ui` is an interactive terminal interface — explore a failing evaluation down to the
resource, assemble a policy from a form, or experiment in a playground. It is an extra rather than
a dependency, because using Tirith as a CI gate should not pay for an interface it never opens.

```bash
pip install 'py-tirith[tui] @ git+https://github.com/StackGuardian/tirith.git'
tirith ui
```

The extra is requested against the git URL for the same reason as above. On Python 3.8 the extra
installs and simply provides nothing.

## Developing against a checkout

```bash
git clone https://github.com/StackGuardian/tirith.git
cd tirith
python -m venv .venv && source .venv/bin/activate
pip install -e .
```
78 changes: 78 additions & 0 deletions .claude/skills/tirith-policies/reference/other-providers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Kubernetes, Infracost and JSON

Choosing a provider is choosing what document you have to feed it. Each names the value it reads
with a **different key** — the wrong key is ignored, not rejected, so the evaluator reads nothing
and the check does not measure what you think.

## Kubernetes

`"required_provider": "stackguardian/kubernetes"`, reading YAML or JSON manifests. Multi-document
YAML is supported.

- `operation_type`: `attribute`
- Requires `kubernetes_kind` — `Pod`, `Deployment`, `Service`, …
- Names the value with **`attribute_path`**

```json
{
"id": "containers_have_liveness_probe",
"provider_args": {
"operation_type": "attribute",
"kubernetes_kind": "Pod",
"attribute_path": "spec.containers.*.livenessProbe"
},
"condition": {"type": "Contains", "value": null, "error_tolerance": 2}
}
```

with `"eval_expression": "!containers_have_liveness_probe"`.

**Why it is written as a detector.** `spec.containers.*.livenessProbe` returns a *list* — one entry
per container, `null` where the probe is missing. `IsNotEmpty` over that list is true as soon as
one container has a probe, which is the wrong question. Test for the presence of `null` and invert.

## Infracost

`"required_provider": "stackguardian/infracost"`, reading an `infracost breakdown --format json`
document.

- `operation_type`: `total_monthly_cost` or `total_hourly_cost`
- `resource_type`: a list. `["*"]` totals everything.

```json
{
"id": "monthly_cost_ceiling",
"provider_args": {"operation_type": "total_monthly_cost", "resource_type": ["*"]},
"condition": {"type": "LessThanEqualTo", "value": 500}
}
```

**The trap: it fails open.** A `resource_type` that matches nothing — a typo, or a type absent from
this plan — sums to `0`, and `0` is less than any ceiling, so the check **passes**. A cost policy
that always passes looks exactly like one that works. Verify against a breakdown that should
exceed the ceiling, and prefer `["*"]` unless you specifically need one type.

## JSON — anything else

`"required_provider": "stackguardian/json"` reads any JSON document: a Terraform state file, an API
response, a CI configuration, a lockfile.

- `operation_type`: `get_value`
- Names the value with **`key_path`**

```json
{
"id": "approval_required",
"provider_args": {"operation_type": "get_value", "key_path": "settings.requireApproval"},
"condition": {"type": "Equals", "value": true}
}
```

`key_path` accepts `*` across a list: `list_of_dicts.*.key1` returns one value per entry, and the
condition is applied to each.

## StackGuardian workflows

`"required_provider": "stackguardian/sg_workflow"` reads a workflow definition, naming the value
with **`workflow_attribute`**, for rules about the pipeline itself rather than the infrastructure —
for example that a Terraform workflow requires approval before apply.
Loading
Loading