The definitive framework for converting codebases into production-ready AI Agent Skills.
SkillFoundry bridges the gap between raw repositories and effective AI agents. It automatically analyzes your codebase, generates an optimized Agent Skill, and—crucially—evaluates its performance against real-world tasks using rigorous benchmarks.
By treating AI skills as software artifacts that require testing, validation, and continuous improvement, SkillFoundry ensures your agents operate reliably and effectively.
- 🧠 Automated Skill Generation: Convert any local directory or GitHub repository into a standardized Agent Skill.
- 📊 Rigorous Evaluation Engine: Automatically generate evaluation tasks to benchmark your skill against baseline agent performance.
- 📈 Data-Driven Improvement: Utilize evaluation failures to autonomously refine and improve the generated skill.
- 🛡️ Security Controls: Built-in secret redaction, SSRF protection, and a constrained process runner. The runner is not a security sandbox.
- 🔌 Multi-Provider Support: Seamless integration with leading LLM providers (OpenAI, Anthropic, Google).
Use this path when you only want to use SkillFoundry:
python3 -m pip install "skillfoundry[openai]"
export OPENAI_API_KEY="your-api-key"
skillfoundry --helpReplace openai with anthropic, google, or all-providers when needed.
Use this copy-paste setup when you want to run the current source checkout:
git clone https://github.com/SinghCod3r/SkillFoundry.git
cd SkillFoundry
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[openai]"
export OPENAI_API_KEY="your-api-key"Build and evaluate a skill from a local project:
skillfoundry build /path/to/your-project --output-dir ./dist
skillfoundry validate ./dist/your-project
skillfoundry eval ./dist/your-project --runs 1Use skillfoundry build --help and skillfoundry eval --help for all options.
Install the checkout with development tools and an LLM provider:
python -m pip install -e ".[dev,openai]"
pytest
ruff check .
pyrightFor a different provider, replace openai in the install command and set its API key environment variable.
SkillFoundry implements a closed-loop system for skill development:
- Analysis: Deep parsing of the target codebase (files, APIs, CLI commands, documentation).
- Generation: Constructing an Agent Skill definition optimized for LLM consumption.
- Benchmarking: Running a baseline agent (without the skill) vs. an enhanced agent (with the skill) across generated tasks.
- Scoring: Utilizing applicable weighted dimensions (Correctness 30%, Success 25%, Safety 20%, Instruction Following 15%, Efficiency 10%). Deterministic judges report only the dimensions they measure; missing dimensions do not silently receive scores.
- Iteration: Automatically fixing skill definitions based on evaluation failures.
| Command | Description |
|---|---|
skillfoundry build <source> |
Analyze a project and generate an Agent Skill |
skillfoundry validate <skill> |
Validate a skill against the AgentSkills specification |
skillfoundry eval <skill> |
Evaluate a skill with a baseline comparison |
skillfoundry improve <skill> |
Improve a skill using feedback from evaluation failures |
skillfoundry compare <a> <b> |
Compare two evaluation result sets |
Append --help to any command for detailed usage.
Control SkillFoundry's behavior using a skillfoundry.toml file in your project root:
[project]
name = "my-custom-skill"
[analysis]
max_files = 5000
max_file_size = 1000000 # in bytes
[evaluation]
runs = 3
timeout = 60 # seconds
[model]
provider = "openai" # Options: openai, anthropic, googleWe welcome contributions from the community! Whether it's adding a new model provider, improving the evaluation engine, or fixing a bug.
Please see our CONTRIBUTING.md for guidelines on how to set up your development environment and submit pull requests.
Security is a primary concern when executing code and handling LLM prompts. Read our SECURITY.md to understand our threat model and how to responsibly report vulnerabilities.
SkillFoundry is open-source software licensed under the Apache 2.0 License.