Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 32 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,19 @@ tasks:
- **Active context + switching** — `devstack use <project>` sets the current
project; with shell integration it `cd`s and sets env in your live shell.
`devstack context` shows where you are; a prompt segment keeps it visible.
- **Framework templates with hot reload** — `node.express`, `node.nestjs`,
`node.next`, `react.vite`, `bun.app`, `turborepo`, `php.laravel.nginx`, plus the
shared engines. App templates bind-mount your source and run the dev server with
file-watch polling for WSL2.
- **51 built-in templates with hot reload** — apps for Go, Rust, Python
(FastAPI/Django/Flask), Node, Bun, Deno, PHP/Laravel, Ruby on Rails, Elixir
Phoenix, Java Spring, ASP.NET, and the Vite family (React, Vue, Svelte, Nuxt,
Astro). App templates bind-mount your source, force the dev server to bind
`0.0.0.0`, and enable file-watch polling for WSL2.
- **Shared engines for most of what you'd reach for** — Postgres, MySQL, MariaDB,
MongoDB, Cassandra, ArangoDB, Redis, Valkey, TimescaleDB, ClickHouse, Neo4j,
MinIO, RustFS, OpenSearch, Meilisearch, Kafka, NATS, RabbitMQ, etcd, Consul,
Keycloak, Jaeger, Mailpit, Mosquitto, LocalStack.
- **Your AI tools learn devstack from devstack** — `devstack ai install` writes
Claude Code skills, a fenced `AGENTS.md` block (read by Codex, Cursor, Copilot,
Gemini CLI, Windsurf, Zed) and an MCP registration into the repo; `devstack ai
mcp` serves the whole CLI over the Model Context Protocol.
- **A task runner** — declare `tasks:` with `deps:` and run the graph with
`devstack run <task>` (dependency-ordered, parallel, streamed).
- **Deterministic generation** — templates render Compose + Dockerfiles
Expand All @@ -111,7 +120,8 @@ tasks:
|---|---|
| **Lifecycle** | `up` · `down` · `status` · `shell` · `run` · `logs` · `dashboard` |
| **Context & DX** | `use` · `context` · `shell-init` · `project list/new` · `env list/set/unset` |
| **Config & templates** | `init` · `config validate/show` · `generate` · `ide` · `template list/lint/test/new` · `import` |
| **Config & templates** | `init` · `config validate/show/schema` · `generate` · `ide` · `template list/lint/test/new` · `import` |
| **AI agents** | `ai install/check` · `ai mcp` · `ai docs` · `ai commands` |
| **Shared & host access** | `shared status/gc/doctor` · `expose` · `ports` |
| **Data plane** | `db` · `s3` · `queue` · `topic` · `stream` · `resource` · `aws -- …` |
| **Multi-repo git** | `ws clone/sync/status/git` |
Expand Down Expand Up @@ -180,17 +190,28 @@ Run `devstack doctor` to check everything.
workspaces, projects, env vars, templates, the data plane, every command, and the
full config reference.
- **[QUICKSTART.md](docs/QUICKSTART.md)** — the 5-minute path.
- **[AI agents](docs/guide/ai-agents.md)** — the `ai` group: MCP, skills, the
embedded docs corpus and the JSON Schemas.
- **[ARCHITECTURE.md](docs/ARCHITECTURE.md)** · **[DECISIONS.md](docs/DECISIONS.md)** · **[ROADMAP.md](docs/ROADMAP.md)** — design, chosen stack, milestones.
- **Component specs** — [`docs/specs/`](docs/specs/) (01…31), each self-contained.
- **Component specs** — [`docs/specs/`](docs/specs/) (01…32), each self-contained.

You can also read the documentation without leaving the terminal — the whole
corpus is compiled into the binary:

```bash
devstack ai docs # list every page
devstack ai docs guide/templates # print one
devstack ai docs --search "port" # search it
```

## Status

🧪 **Beta (0.x).** The shared-services core, the deterministic generation pipeline,
the data plane, multi-repo git, secrets, networking, and the interactive-DX lane
(active context, shell integration, framework/monorepo templates, `devstack run`)
are all implemented and green on `make ci` + `make determinism`. The full
log/dashboard cockpit ([spec 16](docs/specs/16-logs-and-dashboard.md)) is the main
in-flight item.
the data plane, multi-repo git, secrets, networking, the interactive-DX lane
(active context, shell integration, templates, `devstack run`) and the AI-agent
surface ([spec 32](docs/specs/32-ai-agent-integration.md)) are all implemented and
green on `make ci` + `make determinism`. The full log/dashboard cockpit
([spec 16](docs/specs/16-logs-and-dashboard.md)) is the main in-flight item.

## Development

Expand Down
34 changes: 34 additions & 0 deletions docs/guide/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,24 @@ what you need. A missing **required** param fails fast.
| `postgres` | postgres | host, port, user, password, database | 5432 | `version` ("18"), `rootUser` (devstack), `rootPassword` (devstack) |
| `redis` | redis | host, port | 6379 | `version` ("7") |
| `minio` | minio | host, port, accessKey, secretKey | 9000 | `rootUser` (devstackadmin), `rootPassword` (devstackadmin) |
| `mysql` | mysql | host, port, user, password, database | 3306 | `version` ("8"), `rootUser`, `rootPassword` |
| `mariadb` | mariadb | host, port, user, password, database | 3306 | `version`, `rootUser`, `rootPassword` |
| `mongodb` | mongodb | host, port, user, password, database | 27017 | `version`, `rootUser`, `rootPassword` |
| `cassandra` | cassandra | host, port | 9042 | `version` |
| `arangodb` | arangodb | host, port, user, password | 8529 | `version`, `rootPassword` |
| `valkey` | valkey | host, port | 6379 | `version` ("9"); Redis-compatible fork |
| `timescaledb` | timescaledb | host, port, user, password, database | 5432 | `version` ("pg18"); PGDATA is `/home/postgres/pgdata`, not the postgres image's path |
| `clickhouse` | clickhouse | host, port, user, password, database | 8123 | `version`; user/password are REQUIRED or the server refuses network access |
| `neo4j` | neo4j | host, port, user, password, database | 7687 | `rootPassword` (min 8 chars); Bolt on 7687, browser on 7474 |
| `etcd` | etcd | host, port, endpoint | 2379 | `version` (full `vX.Y.Z` — there is no `:latest`), `advertiseHost` |
| `consul` | consul | host, port, endpoint | 8500 | `version`, `logLevel`; `-dev` keeps state in RAM (BUSL-1.1) |
| `rustfs` | rustfs | host, port, accessKey, secretKey, endpoint | 9000 | `image`, `rootUser`, `rootPassword`; S3-compatible, MinIO-shaped defaults |
| `meilisearch` | meilisearch | host, port, password, endpoint | 7700 | `version`, `masterKey`; DB format is tied to the minor version |
| `opensearch` | opensearch | host, port, endpoint | 9200 | `version`, `heapSize`; needs host `vm.max_map_count` >= 262144 |
| `mailpit` | mailpit | host, port, endpoint | 1025 | `version` (leading `v`), `maxMessages`; SMTP on 1025, inbox UI on 8025 |
| `keycloak` | keycloak | host, port, user, password, endpoint | 8080 | `version` (no bare major tag), `rootUser`, `rootPassword` |
| `jaeger` | jaeger | host, port, endpoint | 4317 | `version`; OTLP gRPC 4317, UI 16686, spans held in memory |
| `mosquitto` | mosquitto | host, port | 1883 | `version`; 2.1 ships a working anonymous config, no volume (no persistence) |
| `php.nginx` | — (parent) | — | — | `phpVersion` ("8.3") |
| `php.laravel.nginx` | — (extends `php.nginx`) | — | — | `appEnv` (local) |
| `node.vite` | — | — | — | `nodeVersion` ("20"); runs `npm run dev` |
Expand All @@ -52,6 +70,22 @@ what you need. A missing **required** param fails fast.
| `react.vite` | — | — | — | `nodeVersion` ("20"); Vite HMR |
| `bun.app` | — | — | — | `bunVersion` ("1"); `bun run dev` |
| `turborepo` | — | — | — | `nodeVersion` ("20"); `turbo run dev` (monorepo) |
| `go.app` | — | — | — | `goVersion` ("1.27"), `port`; `air` hot reload with polling |
| `rust.app` | — | — | — | `rustVersion` ("1.98"), `port`; `bacon --headless -j run-long` |
| `python.app` | — (parent) | — | — | `pythonVersion` ("3.13"), `port`; uv + a venv outside the bind mount |
| `python.fastapi` | — (extends `python.app`) | — | — | `appModule`; `uvicorn --reload`, polling forced |
| `python.django` | — (extends `python.app`) | — | — | `allowedHosts`; StatReloader already polls |
| `python.flask` | — (extends `python.app`) | — | — | `appModule`, `port` (5000) |
| `elixir.phoenix` | — | — | — | `elixirVersion`, `port`; **edit `config/dev.exs` to bind 0.0.0.0** |
| `ruby.rails` | — | — | — | `rubyVersion`, `port`; clears the stale `server.pid` on start |
| `java.spring` | — | — | — | `image`, `port`; devtools polls, but a `.java` edit needs a recompile |
| `dotnet.aspnet` | — | — | — | `sdkVersion`, `port`; `dotnet watch --no-launch-profile` |
| `deno.app` | — | — | — | `denoVersion`, `entrypoint`, `port`; no polling fallback |
| `vue.vite` | — | — | — | `nodeVersion` ("24"), `port` (5173) |
| `svelte.kit` | — | — | — | `nodeVersion`, `port` (5173); `npm install` runs `svelte-kit sync` |
| `nuxt` | — | — | — | `nodeVersion`, `port` (3000), `hmrPort` (24678 — Nuxt needs TWO ports) |
| `astro` | — | — | — | `nodeVersion`, `port` (4321 — Astro's default, not Vite's) |
| `smee` | — | — | — | `target`, `channel`; forwards smee.io webhooks inward, binds no port |
| `kafka` (Redpanda) | kafka | host, port, adminPort | 9092 | `image` |
| `nats` | nats | host, port, monitorPort | 4222 | `version` ("2") |
| `rabbitmq` | amqp | host, port, mgmtPort | 5672 | `version` ("3"), `user` (devstack) |
Expand Down
118 changes: 118 additions & 0 deletions internal/cli/app_templates_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
package cli

import (
"strings"
"testing"

"github.com/open-source-cloud/devstack/internal/generate"
"github.com/open-source-cloud/devstack/internal/template"
)

// TestAppTemplatesRenderAndValidate guards the per-language app templates the way
// TestJSTemplatesRenderAndValidate guards the JS ones: each must resolve, validate
// through compose-go, bind-mount the source, and carry a real reload command.
func TestAppTemplatesRenderAndValidate(t *testing.T) {
src := builtinSource()
for _, name := range []string{
"go.app", "rust.app",
"python.app", "python.fastapi", "python.django", "python.flask",
"elixir.phoenix", "ruby.rails", "java.spring", "dotnet.aspnet", "deno.app",
"vue.vite", "svelte.kit", "nuxt", "astro",
} {
res, err := template.Resolve(src, name, nil)
if err != nil {
t.Errorf("%s resolve: %v", name, err)
continue
}
compose, err := generate.LintResolved(name, res)
if err != nil {
t.Errorf("%s lint: %v", name, err)
continue
}
doc := string(compose)
if !strings.Contains(doc, "/app") {
t.Errorf("%s: missing the /app source mount that hot reload depends on:\n%s", name, doc)
}
// An app template must never declare provides: — that is what marks a
// template usable as shared infrastructure.
if res.Provides != "" {
t.Errorf("%s declares provides:%q, but app templates are per-project", name, res.Provides)
}
}
}

// TestAppTemplatesBindAllInterfaces is the single highest-value assertion here.
//
// A dev server bound to 127.0.0.1 inside a container is unreachable from the
// shared network AND from a published host port, and it fails in the most
// confusing way possible: the container is healthy, the port is published, and
// every connection is refused. Each of these frameworks defaults to loopback, so
// the template has to override it explicitly.
func TestAppTemplatesBindAllInterfaces(t *testing.T) {
src := builtinSource()
// name -> a literal proving the template forces an all-interfaces bind.
cases := map[string]string{
"go.app": "0.0.0.0", // HOST/ADDR env the user's code reads
"python.fastapi": "--host 0.0.0.0", // uvicorn defaults to 127.0.0.1
"python.django": "0.0.0.0:", // bare runserver binds 127.0.0.1
"python.flask": "--host 0.0.0.0", // flask run defaults to 127.0.0.1
"ruby.rails": "-b 0.0.0.0", // rails server binds localhost in dev
"dotnet.aspnet": "http://+:", // ASPNETCORE_URLS, plus --no-launch-profile
"vue.vite": "--host 0.0.0.0", // vite server.host defaults to localhost
"svelte.kit": "--host 0.0.0.0", //
"nuxt": "--host 0.0.0.0", //
"astro": "--host 0.0.0.0", // astro server.host defaults to false
"deno.app": "0.0.0.0", // Deno.serve already defaults to 0.0.0.0
}
for name, want := range cases {
res, err := template.Resolve(src, name, nil)
if err != nil {
t.Errorf("%s resolve: %v", name, err)
continue
}
compose, err := generate.LintResolved(name, res)
if err != nil {
t.Errorf("%s lint: %v", name, err)
continue
}
if !strings.Contains(string(compose), want) {
t.Errorf("%s: expected %q so the dev server binds all interfaces; without it the "+
"container looks healthy but refuses every connection:\n%s", name, want, compose)
}
}
}

// TestAppTemplatesSurviveBindMountShadowing checks the dependency directories that
// a `..:/app` bind mount would otherwise hide. Getting this wrong means the image
// built dependencies that the running container then cannot see.
func TestAppTemplatesSurviveBindMountShadowing(t *testing.T) {
src := builtinSource()
cases := map[string][]string{
// The bind mount clobbers whatever the build put at these paths, so each
// needs its own volume.
"elixir.phoenix": {"/app/deps", "/app/_build"},
"dotnet.aspnet": {"/app/obj", "/app/bin"},
"vue.vite": {"/app/node_modules"},
"svelte.kit": {"/app/node_modules"},
"nuxt": {"/app/node_modules"},
"astro": {"/app/node_modules"},
}
for name, paths := range cases {
res, err := template.Resolve(src, name, nil)
if err != nil {
t.Errorf("%s resolve: %v", name, err)
continue
}
compose, err := generate.LintResolved(name, res)
if err != nil {
t.Errorf("%s lint: %v", name, err)
continue
}
for _, p := range paths {
if !strings.Contains(string(compose), p) {
t.Errorf("%s: %s is not shielded from the bind mount, so the container "+
"cannot see what the image build installed there", name, p)
}
}
}
}
102 changes: 102 additions & 0 deletions internal/cli/template_docs_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package cli

import (
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"testing"

"github.com/open-source-cloud/devstack/internal/template"
"github.com/open-source-cloud/devstack/templates"
)

// docRowRE pulls the leading backticked name out of a markdown table row.
var docRowRE = regexp.MustCompile("^\\|\\s*`([^`]+)`")

// repoFile reads a path relative to the repository root.
func repoFile(t *testing.T, rel string) string {
t.Helper()
b, err := os.ReadFile(filepath.Join("..", "..", filepath.FromSlash(rel)))
if err != nil {
t.Fatalf("read %s: %v", rel, err)
}
return string(b)
}

// documentedTemplates returns the template names a markdown file tabulates,
// limited to names that are actually built-in (the same files carry unrelated
// tables for template functions, flags and registry commands).
func documentedTemplates(doc string, builtin map[string]bool) map[string]bool {
out := map[string]bool{}
for _, line := range strings.Split(doc, "\n") {
m := docRowRE.FindStringSubmatch(strings.TrimSpace(line))
if m == nil {
continue
}
name := strings.TrimSpace(m[1])
if builtin[name] {
out[name] = true
}
}
return out
}

// TestBuiltinTemplatesAreDocumented keeps the two hand-maintained template tables
// honest against the embedded set.
//
// Both tables had silently gone stale before this test existed: templates/README.md
// was missing every spec-28 engine (kafka, nats, rabbitmq, localstack, ministack)
// and docs/guide/templates.md was missing the mysql/mariadb/mongodb/cassandra/
// arangodb batch. Adding a template without a row is now a test failure rather
// than something noticed months later.
func TestBuiltinTemplatesAreDocumented(t *testing.T) {
src := template.NewFSSource(templates.FS)
names := src.List()
if len(names) < 40 {
t.Fatalf("expected the full built-in set, got %d", len(names))
}
builtin := make(map[string]bool, len(names))
for _, n := range names {
builtin[n] = true
}

for _, f := range []string{"templates/README.md", "docs/guide/templates.md"} {
documented := documentedTemplates(repoFile(t, f), builtin)
var missing []string
for _, n := range names {
if !documented[n] {
missing = append(missing, n)
}
}
sort.Strings(missing)
for _, n := range missing {
t.Errorf("%s has no row for the built-in template %q", f, n)
}
}
}

// TestEveryBuiltinTemplateIsEmbedded catches the other half of the same mistake:
// a template directory that exists on disk but was never added to the go:embed
// allow-list in templates/embed.go is invisible to the binary, so it lints from a
// directory path and then does not exist for any real workspace.
func TestEveryBuiltinTemplateIsEmbedded(t *testing.T) {
entries, err := os.ReadDir(filepath.Join("..", "..", "templates"))
if err != nil {
t.Fatalf("read templates dir: %v", err)
}
embedded := map[string]bool{}
for _, n := range template.NewFSSource(templates.FS).List() {
embedded[n] = true
}
for _, e := range entries {
if !e.IsDir() {
continue
}
if !embedded[e.Name()] {
t.Errorf("templates/%s exists on disk but is not in the go:embed list in "+
"templates/embed.go, so the binary cannot see it", e.Name())
}
}
}
Loading
Loading