Skip to content

Latest commit

Β 

History

14 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš‘ AutoSRE - Autonomous API Healing Platform

AutoSRE is an advanced Autonomous Site Reliability Engineering (SRE) platform designed to detect, diagnose, and automatically remediate failures in microservice-based systems.

By combining deterministic observability, generative AI, and machine learning, AutoSRE acts as an automated Level 1 / Level 2 SRE responder, enabling self-healing infrastructure with minimal human intervention.


🌟 Features

Zero-Touch Auto-Remediation

AutoSRE automatically:

  • Patch source code bugs
  • Restart unhealthy containers
  • Scale databases and services
  • Flush caches
  • Recover from configuration drift

All actions are driven by real-time logs, stack traces, and metrics.


Dual-Plane Architecture

βš™οΈ Control Plane (Java / Spring Boot)

Responsible for:

  • Service orchestration
  • Docker container management
  • Safety policy enforcement
  • Synthetic traffic verification
  • Applying code patches

Runs on:

Port 8080

🧠 Intelligence Plane (Python / FastAPI)

Contains the AI brain:

  • LangGraph repair loops
  • Gemini-based Root Cause Analysis (RCA)
  • ML anomaly detection
  • Retrieval-Augmented Memory (RAG)
  • Confidence scoring

Runs on:

Port 8000

Predictive Anomaly Detection

Uses Scikit-Learn models:

  • Isolation Forest
  • Linear Regression

to proactively detect:

  • Memory leaks
  • CPU spikes
  • Slow-burn degradation
  • Future failures before outages occur

Episodic Memory (RAG)

Powered by ChromaDB.

Stores:

  • Historical incidents
  • Previous fixes
  • Success rates
  • Decision confidence

This enables AutoSRE to continuously learn from past outages.


AST-Validated Code Surgeon

LLM-generated patches are validated using Python's Abstract Syntax Tree (AST) parser before deployment, ensuring syntactic correctness and reducing bad repairs.


Built-In Chaos Engineering

AutoSRE ships with intentionally vulnerable microservices for testing and demonstrations.


πŸ— Architecture

                +---------------------------+
                |       Prometheus          |
                +------------+--------------+
                             |
                             v
+------------------------------------------------------+
|                Control Plane (Java)                  |
|------------------------------------------------------|
| CrashWatcher                                          |
| Policy Engine                                         |
| Healing Executor                                      |
| Ghost Traffic Verifier                                |
+----------------------+-------------------------------+
                       |
                       | Analyze Failure
                       v
+------------------------------------------------------+
|             Intelligence Plane (Python)              |
|------------------------------------------------------|
| RAG Memory (ChromaDB)                                |
| RCA Engine (Gemini)                                  |
| LangGraph Doctor β†’ Router β†’ Surgeon β†’ Validator      |
| ML Anomaly Detection                                 |
+----------------------+-------------------------------+
                       |
                       | Repair Decision
                       v
+------------------------------------------------------+
|                 Victim Microservices                 |
+------------------------------------------------------+

πŸ“ Project Structure

godn76-autosre/
β”‚
β”œβ”€β”€ bootstrap.sh
β”‚
β”œβ”€β”€ Global-AutoSRE/
β”‚   β”œβ”€β”€ agents/
β”‚   β”œβ”€β”€ ml/
β”‚   └── main.py
β”‚
β”œβ”€β”€ Local-AutoSRE/
β”‚   β”œβ”€β”€ src/main/java/
β”‚   β”‚   β”œβ”€β”€ watcher/
β”‚   β”‚   β”œβ”€β”€ executor/
β”‚   β”‚   β”œβ”€β”€ policy/
β”‚   β”‚   └── verifier/
β”‚   └── pom.xml
β”‚
└── VICTIMS/
    β”œβ”€β”€ victim-python/
    β”œβ”€β”€ victim-node/
    β”œβ”€β”€ victim-memory-leak/
    β”œβ”€β”€ victim-config-drift/
    └── victim-unfixable/

πŸ”„ Auto-Healing Workflow

1. Detect

CrashWatcher continuously monitors Docker logs for:

  • Exception
  • Error
  • Traceback

2. Analyze

AutoSRE bundles:

  • Stack traces
  • Recent logs
  • Prometheus metrics

and sends them to the Intelligence Plane.


3. Diagnose & Decide

The AI brain:

  1. Queries historical incidents from ChromaDB.
  2. Performs root cause analysis.
  3. Determines the appropriate action.

Possible actions:

  • CODE_FIX
  • RESTART
  • SCALE
  • CACHE_FLUSH
  • NO_ACTION

4. Repair

If the failure is code-related:

Doctor Node
      ↓
Router Node
      ↓
Surgeon Node
      ↓
Validator Node

The Validator checks the generated patch using AST before approval.


5. Execute

The Control Plane:

  • Applies the patch
  • Updates host files
  • Restarts containers

using Docker commands.


6. Verify & Learn

GhostTrafficVerifier sends synthetic requests to verify service health.

Results are fed back into ChromaDB to improve future decision-making.


πŸ§ͺ Chaos Testing Environment

The platform includes several deliberately broken services.

victim-python

Produces:

KeyError

victim-node

Produces:

TypeError

victim-memory-leak

Simulates:

  • Memory exhaustion
  • Gradual degradation

victim-config-drift

Simulates:

  • Invalid configurations
  • Misaligned environments

victim-unfixable

Represents severe failures such as:

  • Database integrity violations
  • Corrupted state

These require human intervention.


πŸš€ Getting Started

Prerequisites

Docker & Docker Compose

Required for:

  • Victim services
  • Prometheus
  • Intelligence Plane

Java 21+

Required for:

  • Spring Boot Control Plane

Python 3.10+

Required for:

  • FastAPI
  • LangGraph
  • ML models

Node.js (Optional)

Useful if a frontend UI is added later.


βš™ Configuration

Navigate to:

Global-AutoSRE/

Copy the environment file:

cp .env.example .env

Add your Gemini API key:

GEMINI_API_KEY=your_gemini_api_key_here

β–Ά Startup

Make the bootstrap script executable:

chmod +x bootstrap.sh

Run:

./bootstrap.sh

The script automatically:

  • Verifies dependencies
  • Checks port availability
  • Builds Spring Boot
  • Starts Docker containers
  • Launches background services

πŸ”Œ Port Mapping

Port Service Description
8080 Spring Boot Control Plane Main orchestrator API
8000 Python Intelligence Plane AI APIs and ML models
9090 Prometheus Metrics and observability
8001-8005 Chaos Victims Vulnerable microservices

πŸ›‘ Stopping AutoSRE

Find the Spring Boot process:

ps aux | grep Local-AutoSRE

Terminate it:

kill <PID>

Stop Docker services:

docker compose -f VICTIMS/docker-compose.yml down
docker compose -f Global-AutoSRE/docker-compose.yml down

πŸ›‘ Safety & Policy Engine

AutoSRE contains built-in guardrails to avoid unstable behavior.

Rate Limiting

Repeated crashes are throttled to prevent repair loops.


Maximum Restart Count

Services may be restarted at most:

3 times per policy window

Execution Modes

AUTO

Fully autonomous healing.

Detect β†’ Diagnose β†’ Repair β†’ Verify

MANUAL

Human approval is required before execution.


OBSERVABILITY_ONLY

AutoSRE acts solely as an advisor.

No actions are performed automatically.


🧠 Technology Stack

Control Plane

  • Java 21
  • Spring Boot
  • Maven
  • Docker

Intelligence Plane

  • Python 3.10+
  • FastAPI
  • LangGraph
  • Google Gemini
  • ChromaDB
  • Scikit-Learn

Observability

  • Prometheus

Containerization

  • Docker
  • Docker Compose

Machine Learning

  • Isolation Forest
  • Linear Regression

🎯 Vision

AutoSRE aims to transform traditional monitoring systems into autonomous reliability platforms that can:

  • Detect failures
  • Understand root causes
  • Repair systems
  • Verify recovery
  • Learn from every incident

ultimately enabling self-healing microservice infrastructures.


License

MIT License


AutoSRE β€” Your Autonomous SRE Engineer.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages