Skip to content

feat(troubleshoot): realistic incident recovery scenarios and deterministic verification - #13

Merged
howlcipher merged 1 commit into
mainfrom
feat/troubleshooting-recovery-scenarios
Aug 19, 2026
Merged

howlcipher merged 1 commit into
mainfrom
feat/troubleshooting-recovery-scenarios

Conversation

@howlcipher

Copy link
Copy Markdown
Owner

Objective

Implement realistic troubleshooting scenarios with deterministic recovery verification in DevOps-Learn-by-Doing.

Summary of Changes

  • Core Troubleshooting Scenarios:
    • port_conflict: Port binding collision (EADDRINUSE). Distinguishes host port collisions from container internal errors, verifies socket reallocation and /health reachability.
    • missing_config: Missing required environment configuration (fail-fast startup). Distinguishes container launch from initialization crash, verifies environment injection and valid startup.
    • health_check_failure: Degraded internal dependency check (process alive != service healthy). Distinguishes running container process from degraded HTTP 503 probe response, verifies healthy state resolution.
    • resource_limit: Bounded container memory limit constraint triggering exit code 137 (SIGKILL/OOMKilled) without application traceback. Teaches OOM diagnosis and adequate memory threshold allocation.
  • Scenario Lifecycle & Engine:
    • Enforces explicit lifecycle: SETUP -> INJECT -> OBSERVE -> EXPLAIN -> REMEDIATE -> VERIFY -> CLEANUP.
    • Distinguishes factual observations from interpretive hypotheses.
    • Progressive assistance (Levels 0-4) provides evidence-first learning without prematurely leaking remediation.
    • Deterministic recovery verification checks actual post-remediation system health.
    • Guaranteed finally teardown cleans up sockets, containers, and test state.
  • CLI & Workflows:
    • devops-learn troubleshoot list, devops-learn troubleshoot run <scenario>, and devops-learn troubleshoot doctor.
    • Honest reporting of LIVE VERIFIED vs SIMULATED / TESTED.
  • Quality & Falsification:
    • Comprehensive negative tests and falsification suite proving that wrong remediations, empty inputs, degraded dependencies, and undersized memory limits fail verification.
    • All 170 unit and integration tests passing. Full lint and type check suite clean.

@howlcipher
howlcipher merged commit 5519852 into main Aug 19, 2026
3 checks passed
@howlcipher
howlcipher deleted the feat/troubleshooting-recovery-scenarios branch August 19, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant