Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/field-test-axum-v0.2.6.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# StackPilot v0.2.6 — Rust/Axum field-test wrap-up

Date: 2026-09-15

## Scope

This field test exercised StackPilot v0.2.6 against an existing Rust/Axum repository on Windows x86_64. The goal was to validate the adoption path end to end rather than only verify generated-project scaffolding.

## Result

The benchmark completed successfully at **100/100 readiness-v1** after explicit security and AWS Terraform remediation.

Validated behavior:

- Windows installer and current-shell CLI usage
- Rust and Axum detection
- Docker and Compose detection
- GitHub Actions detection
- environment hygiene detection
- health/readiness endpoint detection
- safe `fix` preview behavior
- `.stackpilot.toml` adoption
- generic `fix --apply` idempotency
- explicit `--security` remediation
- ecosystem-aware Cargo + GitHub Actions Dependabot configuration
- dependency, secret, SBOM and container scanning detection
- GitHub Actions explicit-permissions detection
- security-remediation idempotency
- explicit `--cloud AWS` Terraform generation
- Terraform-remediation idempotency
- `terraform fmt -check`
- `terraform init`
- `terraform validate`

## Bugs confirmed by the field test

### 1. Adopted metadata could become stale after cloud remediation

Observed sequence:

1. adopt an existing repository, creating `.stackpilot.toml` with `cloud = "None"` and `terraform = false`;
2. later run `stackpilot fix . --cloud AWS --apply`;
3. Terraform is generated and readiness detects it, but the previously created metadata remains stale.

Wrap-up fix:

- explicit cloud remediation now synchronizes an existing regular `.stackpilot.toml` after Terraform is successfully present;
- the cloud value is normalized to `AWS`, `Azure`, or `GCP`;
- `[features].terraform` is synchronized to `true`;
- preview mode never writes metadata and reports the synchronization separately.

### 2. Terraform variable descriptions could expose an unrelated application package name

The Rust adapter derives an application package identity from `Cargo.toml`. In an adopted repository this can differ from the repository identity and produced descriptions such as `AWS region for stellarsend-backend` during an unrelated field test.

Wrap-up fix:

- provider variable descriptions are now identity-neutral and describe the generated StackPilot Terraform foundation instead of embedding an application/package name.

## Deferred to a later work session

The following are intentionally documented but not included in this wrap-up scope:

- make `stackpilot doctor` context-aware so it can report missing optional tools such as Terraform when they are relevant to the repository;
- design `readiness-v2` so supply-chain controls can contribute appropriate weight rather than remaining only additional findings;
- distinguish provider-foundation Terraform from meaningful deployable infrastructure when assigning infrastructure readiness;
- evaluate safe preview-first remediation for existing GitHub Actions workflows that lack explicit permissions, without blindly changing third-party or write-capable workflows;
- run the next cross-stack adoption benchmark, with TypeScript/NestJS as the preferred next target.

## Release/benchmark interpretation

A `100/100` readiness-v1 score means every control represented by the current model was detected. It should not be interpreted as a claim that the application has complete production infrastructure or that every possible supply-chain control has been implemented.

This document closes the Rust/Axum v0.2.6 field-test session. Further enhancements above should be handled separately so the benchmark remains reproducible and the wrap-up change stays narrowly scoped.
8 changes: 4 additions & 4 deletions recipes/base/templates/terraform/variables.tf.j2
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{% if cloud == "AWS" %}variable "region" {
description = "AWS region for {{ project_name }}"
description = "AWS region for the generated StackPilot Terraform foundation"
type = string
default = "us-east-1"
}
{% elif cloud == "Azure" %}variable "location" {
description = "Azure location for {{ project_name }}"
description = "Azure location for the generated StackPilot Terraform foundation"
type = string
default = "eastus"
}
{% elif cloud == "GCP" %}variable "project_id" {
description = "GCP project ID for {{ project_name }}"
description = "GCP project ID for the generated StackPilot Terraform foundation"
type = string
}

variable "region" {
description = "GCP region for {{ project_name }}"
description = "GCP region for the generated StackPilot Terraform foundation"
type = string
default = "us-central1"
}
Expand Down
2 changes: 2 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ mod fix;
mod git;
mod identity;
mod inspect;
mod metadata;
mod readiness;
mod recipe;
mod scaffold;
Expand Down Expand Up @@ -333,6 +334,7 @@ fn main() -> Result<()> {
deployment.as_deref(),
apply,
)?;
metadata::sync_after_fix(&path, cloud.as_deref(), apply)?;
}
Commands::Upgrade {
path,
Expand Down
166 changes: 166 additions & 0 deletions src/metadata.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
use std::{fs, path::Path};

use anyhow::{Context, Result};

pub fn sync_after_fix(root: &Path, cloud: Option<&str>, apply: bool) -> Result<()> {
let Some(cloud) = normalize_cloud(cloud) else {
return Ok(());
};

let root = root
.canonicalize()
.with_context(|| format!("failed to resolve repository path {}", root.display()))?;
let metadata_path = root.join(".stackpilot.toml");
if !metadata_path.exists() {
return Ok(());
}

let metadata = fs::symlink_metadata(&metadata_path)
.with_context(|| format!("failed to inspect {}", metadata_path.display()))?;
if metadata.file_type().is_symlink() || !metadata.file_type().is_file() {
println!(
"\n! StackPilot metadata synchronization skipped because {} is not a regular file",
metadata_path.display()
);
return Ok(());
}

let original = fs::read_to_string(&metadata_path)
.with_context(|| format!("failed to read {}", metadata_path.display()))?;
let Some(content) = synchronized_content(&original, cloud) else {
println!(
"\n! StackPilot metadata synchronization skipped because .stackpilot.toml does not contain the expected [project].cloud and [features].terraform keys"
);
return Ok(());
};

if content == original {
return Ok(());
}

let terraform_exists = root.join("infra/terraform/main.tf").is_file();
if !apply {
println!("\nStackPilot metadata synchronization");
println!(
"~ update .stackpilot.toml — synchronize explicit cloud intent ({cloud}) and Terraform feature metadata"
);
if !terraform_exists {
println!(" conditional on Terraform remediation being applied successfully");
}
return Ok(());
}

if !terraform_exists {
return Ok(());
}

fs::write(&metadata_path, content)
.with_context(|| format!("failed to update {}", metadata_path.display()))?;
println!("\n✓ StackPilot metadata synchronized: cloud={cloud}, terraform=true");
Ok(())
}

fn normalize_cloud(cloud: Option<&str>) -> Option<&'static str> {
match cloud {
Some(value) if value.eq_ignore_ascii_case("AWS") => Some("AWS"),
Some(value) if value.eq_ignore_ascii_case("Azure") => Some("Azure"),
Some(value) if value.eq_ignore_ascii_case("GCP") => Some("GCP"),
_ => None,
}
}

fn synchronized_content(content: &str, cloud: &str) -> Option<String> {
let content = replace_section_key(content, "project", "cloud", &format!("\"{cloud}\""))?;
replace_section_key(&content, "features", "terraform", "true")
}

fn replace_section_key(content: &str, section: &str, key: &str, value: &str) -> Option<String> {
let newline = if content.contains("\r\n") {
"\r\n"
} else {
"\n"
};
let had_trailing_newline = content.ends_with('\n');
let target_section = format!("[{section}]");
let target_key = format!("{key} =");
let mut current_section = "";
let mut replaced = false;
let mut lines = Vec::new();

for line in content.lines() {
let trimmed = line.trim();
if trimmed.starts_with('[') && trimmed.ends_with(']') {
current_section = trimmed;
}

if current_section == target_section && trimmed.starts_with(&target_key) {
let indent_len = line.len() - line.trim_start().len();
let indent = &line[..indent_len];
lines.push(format!("{indent}{key} = {value}"));
replaced = true;
} else {
lines.push(line.to_string());
}
}

if !replaced {
return None;
}

let mut result = lines.join(newline);
if had_trailing_newline {
result.push_str(newline);
}
Some(result)
}

#[cfg(test)]
mod tests {
use std::fs;

use tempfile::tempdir;

use super::{sync_after_fix, synchronized_content};

const METADATA: &str = "version = 1\n\n[project]\nname = \"demo\"\ncloud = \"None\"\n\n[features]\ndocker = true\nci = true\nterraform = false\n\n[stackpilot]\nrecipe = \"adopted\"\nmanaged = false\n";

#[test]
fn synchronizes_cloud_and_terraform_without_rewriting_other_metadata() {
let updated = synchronized_content(METADATA, "AWS").expect("metadata update");
assert!(updated.contains("name = \"demo\""));
assert!(updated.contains("cloud = \"AWS\""));
assert!(updated.contains("terraform = true"));
assert!(updated.contains("managed = false"));
}

#[test]
fn apply_updates_adopted_metadata_after_terraform_exists() {
let repo = tempdir().expect("repository");
fs::write(repo.path().join(".stackpilot.toml"), METADATA).expect("metadata");
fs::create_dir_all(repo.path().join("infra/terraform")).expect("terraform directory");
fs::write(
repo.path().join("infra/terraform/main.tf"),
"terraform {}\n",
)
.expect("terraform main");

sync_after_fix(repo.path(), Some("aws"), true).expect("sync metadata");

let updated = fs::read_to_string(repo.path().join(".stackpilot.toml")).expect("metadata");
assert!(updated.contains("cloud = \"AWS\""));
assert!(updated.contains("terraform = true"));
}

#[test]
fn preview_never_modifies_metadata() {
let repo = tempdir().expect("repository");
fs::write(repo.path().join(".stackpilot.toml"), METADATA).expect("metadata");

sync_after_fix(repo.path(), Some("AWS"), false).expect("preview metadata");

assert_eq!(
fs::read_to_string(repo.path().join(".stackpilot.toml")).expect("metadata"),
METADATA
);
}
}
Loading