Skip to content
Merged
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,17 @@ A **capability** is a connection the environment exposes; a **harness** attaches

From the [platform UI](https://hud.ai) you can run batches, compare models on the same taskset, and inspect every trace.

→ [Run & deploy](https://docs.hud.ai/v6/reference/runtime)
A **project** holds the environments and tasksets a team creates and decides who can see them. Deploying without one uses your team's default project, so nothing here is required to get started. To put an environment somewhere else, pin the directory once and every later `hud deploy` and `hud sync tasks` follows it:

```bash
hud project list # projects you can see
hud project use browser-evals # writes projectId to .hud/config.json
hud project # where does a deploy here land?
```

`hud deploy` and `hud sync tasks` also take `--project <name-or-id>` as a one-command override; it does not change the directory configuration. Use `hud set HUD_DEFAULT_PROJECT=<name-or-id>` for a machine-wide fallback. Precedence is the flag, then the directory's `.hud/config.json`, then `HUD_DEFAULT_PROJECT`, then your team default. An environment or taskset that already exists stays where it is; naming a different project fails rather than moving it.

→ [Projects](https://docs.hud.ai/v6/reference/projects) · [Run & deploy](https://docs.hud.ai/v6/reference/runtime)

## Train on rewards

Expand Down
2 changes: 1 addition & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"groups": [
{ "group": "Start here", "pages": ["v6/start/index", "v6/start/quickstart", "v6/start/overview"] },
{ "group": "Guides", "pages": ["v6/guides/creating-an-environment", "v6/guides/running-an-eval", "v6/guides/training-agents"] },
{ "group": "Reference", "pages": ["v6/reference/environment", "v6/reference/tasks", "v6/reference/capabilities", "v6/reference/agents", "v6/reference/runtime", "v6/reference/graders", "v6/reference/advice", "v6/reference/training", "v6/reference/types", "v6/reference/cli", "v6/reference/telemetry"] },
{ "group": "Reference", "pages": ["v6/reference/environment", "v6/reference/tasks", "v6/reference/projects", "v6/reference/capabilities", "v6/reference/agents", "v6/reference/runtime", "v6/reference/graders", "v6/reference/advice", "v6/reference/training", "v6/reference/types", "v6/reference/cli", "v6/reference/telemetry"] },
{ "group": "Advanced", "pages": [
{ "group": "Advanced", "expanded": false, "pages": ["v6/advanced/extending", "v6/advanced/robots"] }
] },
Expand Down
13 changes: 13 additions & 0 deletions docs/v6/guides/creating-an-environment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -289,10 +289,23 @@ hud set HUD_API_KEY=your-key # get one at hud.ai
hud deploy
```

The environment is created in the directory's resolved [HUD Project](/v6/reference/projects), or in
the team default Project when none is configured. Select another Project persistently or on this
deploy:

```bash
hud project use browser-evals
hud deploy

# Or override this deploy without changing the directory Project
hud deploy --project browser-evals
```

To run a taskset from the platform, publish it too. `hud sync` uploads a taskset and only what changed:

```bash
hud sync tasks my-tasks # publish tasks.py as a named taskset
hud sync tasks my-tasks --project browser-evals
hud sync env # sync environment metadata
```

Expand Down
60 changes: 57 additions & 3 deletions docs/v6/reference/cli.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "CLI"
description: "Reference for the hud command-line interface: init, dev, run, eval, models, and other commands that span the HUD environment and agent lifecycle."
description: "Reference for the hud command-line interface: init, deploy, projects, sync, eval, and other commands across the HUD lifecycle."
icon: "terminal"
---

Expand Down Expand Up @@ -64,13 +64,15 @@ rebuilds that environment.

```bash
hud deploy
hud deploy --project browser-evals
```

| Option | Description |
|--------|-------------|
| `--all`, `-a` | Deploy all environments in the directory. |
| `--env`, `-e` | Env var `KEY=VALUE` (repeatable). |
| `--env-file` | Path to a `.env` file. |
| `--project` | One-command Project override by name or ID. Falls back to the directory Project, `HUD_DEFAULT_PROJECT`, then the team default. |

## Evaluate

Expand Down Expand Up @@ -140,11 +142,63 @@ hud task grade fix_bug --source tasks.py --url tcp://127.0.0.1:8765 --answer "..

## Platform

### `hud project`

Show and choose the [HUD Project](/v6/reference/projects) where new environments and tasksets are
created.

```bash
hud sync tasks my-taskset # publish tasks as a named taskset
hud sync env # sync environment metadata
hud project # resolved Project for the current directory
hud project list # visible Projects and access
hud project create browser-evals # create and use a Project (team admins)
hud project create browser-evals --no-use
hud project use browser-evals # save projectId to .hud/config.json
hud project use browser-evals -C ./my-env
```

| Command or option | Description |
|-------------------|-------------|
| `hud project [--directory PATH]` | Show where new resources from a directory will be created. `-C` is an alias. |
| `hud project list` | List visible Projects, marking the team default and read-only entries. |
| `hud project create <name>` | Create and pin a Project. Accepts `--description`, `--directory`/`-C`, and `--no-use`. |
| `hud project use <name-or-id>` | Pin a directory to an existing writable Project. Accepts `--directory`/`-C`. |

#### Project selection

`hud deploy` and `hud sync tasks` resolve their destination in this order:

| Priority | Source | Scope |
|----------|--------|-------|
| 1 | `--project <name-or-id>` | Current command |
| 2 | `.hud/config.json` → `projectId` | Current directory |
| 3 | `HUD_DEFAULT_PROJECT` | Machine-wide fallback |
| 4 | Team default Project | Zero-configuration fallback |

`--project` does not change `.hud/config.json`. Use `hud project use <name-or-id>` when later deploys
and task syncs from the directory should use that Project. Set a fallback for unconfigured
directories with `hud set HUD_DEFAULT_PROJECT=<name-or-id>`.

`hud project list` marks the team default and read-only Projects. Pinning a directory, deploying, or
syncing to a Project requires create access. Only team admins can create Projects.

An existing environment or taskset stays in its original Project. Selecting a different Project for
an existing resource fails rather than moving it; use its current Project or create a separately named
resource in the new one.

### `hud sync`

Publish task definitions or link a local directory to a deployed environment:

```bash
hud sync tasks my-taskset # publish using resolved Project
hud sync tasks my-taskset --project browser-evals # override by Project name or ID
hud sync env # link environment metadata
```

`hud sync tasks --project` is a one-command override, like `hud deploy --project`. Existing resources
remain in their original Project. See
[Projects](/v6/reference/projects) for the conceptual overview.

External benchmark formats can be adapted into runnable `Taskset`s through the
experimental [Harbor integration](/v6/experimental/harbor).

Expand Down
42 changes: 42 additions & 0 deletions docs/v6/reference/projects.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
title: "Projects"
description: "Understand how HUD Projects organize environments and tasksets."
icon: "folder-tree"
---

A HUD **Project** is the workspace boundary for a related set of environments and tasksets. Projects
also control who can see those resources and who can create new ones.

Projects is currently a limited beta. If it is not enabled for your team, Project-specific CLI
commands stop with a message explaining how to request access; normal deploy and task sync workflows
continue to use the team default Project.

## Default Project

Every team has a default Project. If you deploy an environment or publish a taskset without choosing
another Project, HUD places it in the team default. This keeps the normal workflow zero-configuration.

## Project selection

A local environment directory can be associated with a Project. Deploys and task syncs from that
directory then use the same destination, keeping an environment and its tasksets together.

For automation or exceptional cases, a command can select a different destination explicitly without
changing the directory association. A machine-wide preference can provide a fallback for directories
without an association.

Project selection only controls where a **new** resource is created. It does not move resources
between Projects.

## Resource ownership

Each deployed environment or taskset belongs to one Project. If you need the same resource in another
Project, create it there under a separate name rather than changing the Project of the existing
resource.

Projects may be writable or read-only for a given user. Only team admins can create Projects.

<CardGroup cols={2}>
<Card title="Projects CLI" icon="terminal" href="/v6/reference/cli#hud-project" />
<Card title="Tasks & tasksets" icon="list-check" href="/v6/reference/tasks" />
</CardGroup>
13 changes: 13 additions & 0 deletions docs/v6/reference/tasks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,19 @@ uploads a taskset and only what changed, a workflow covered in
[creating an environment](/v6/guides/creating-an-environment#deploying-to-the-platform). In code,
`diff(local, remote)` returns a `SyncPlan` describing the comparison:

To create the taskset outside the team default Project, select a directory Project first or pass an
explicit destination:

```bash
hud project use browser-evals
hud sync tasks my-taskset

# Or select it on the sync command
hud sync tasks my-taskset --project browser-evals
```

See [Projects](/v6/reference/projects) for placement precedence and existing-resource behavior.

```python
from hud.eval.sync import diff

Expand Down
2 changes: 2 additions & 0 deletions hud/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
from .init import init_command # noqa: E402
from .jobs import jobs_app # noqa: E402
from .models import models_app # noqa: E402
from .project import project_app # noqa: E402
from .qa import qa_app # noqa: E402
from .serve import serve_command # noqa: E402
from .sync import sync_app # noqa: E402
Expand All @@ -52,6 +53,7 @@
app.add_typer(jobs_app, name="jobs")
app.add_typer(trace_app, name="trace")
app.add_typer(qa_app, name="qa")
app.add_typer(project_app, name="project")


@app.command(name="set")
Expand Down
17 changes: 17 additions & 0 deletions hud/cli/deploy.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from hud.cli.utils.build_logs import poll_build_status, stream_build_logs
from hud.cli.utils.config import parse_env_file, parse_key_value
from hud.cli.utils.context import create_build_context_tarball, format_size
from hud.cli.utils.project import PROJECT_OPTION_HELP, Placement, resolve_writable_placement
from hud.cli.utils.registry import get_registry_environment
from hud.cli.utils.source import EnvironmentSource
from hud.eval.runtime import ComposeProject, RuntimeConfig
Expand All @@ -41,6 +42,7 @@
class _DeployPlan:
name: str
registry_id: str | None
placement: Placement
runtime: str | None
runtime_config: RuntimeConfig | None
env_vars: dict[str, str]
Expand Down Expand Up @@ -375,6 +377,7 @@ def _prepare_deploy_plan(
env_file: str | None,
no_env: bool,
registry_id: str | None,
project: str | None,
build_args: list[str] | None,
build_secrets: list[str] | None,
runtime: str | None,
Expand All @@ -390,6 +393,7 @@ def _prepare_deploy_plan(
platform,
console,
)
placement = resolve_writable_placement(platform, env_source, flag=project, console=console)
skip_dotenv = _skip_dotenv(
env_source,
env_dir,
Expand Down Expand Up @@ -437,6 +441,7 @@ def _prepare_deploy_plan(
return _DeployPlan(
name=resolved_name,
registry_id=registry_id,
placement=placement,
runtime=normalized_runtime,
runtime_config=loaded_runtime_config,
env_vars=env_vars,
Expand All @@ -453,6 +458,7 @@ def deploy_environment(
no_cache: bool = False,
verbose: bool = False,
registry_id: str | None = None,
project: str | None = None,
build_args: list[str] | None = None,
build_secrets: list[str] | None = None,
runtime: str | None = None,
Expand Down Expand Up @@ -491,6 +497,7 @@ def deploy_environment(
env_file=env_file,
no_env=no_env,
registry_id=registry_id,
project=project,
build_args=build_args,
build_secrets=build_secrets,
runtime=runtime,
Expand Down Expand Up @@ -555,6 +562,7 @@ async def _trigger_build(
key: value
for key, value in (
("registry_id", plan.registry_id),
("project_id", plan.placement.project_id),
("runtime_provider", plan.runtime),
(
"runtime_config",
Expand Down Expand Up @@ -715,6 +723,7 @@ def deploy_all(
no_env: bool = False,
no_cache: bool = False,
verbose: bool = False,
project: str | None = None,
build_args: list[str] | None = None,
build_secrets: list[str] | None = None,
runtime: str | None = None,
Expand Down Expand Up @@ -755,6 +764,7 @@ def deploy_all(
no_cache=no_cache,
verbose=verbose,
registry_id=None,
project=project,
build_args=build_args,
build_secrets=build_secrets,
runtime=runtime,
Expand Down Expand Up @@ -833,6 +843,11 @@ def deploy_command(
help="Existing registry ID for rebuilds (advanced)",
hidden=True,
),
project: str | None = typer.Option(
None,
"--project",
help=PROJECT_OPTION_HELP,
),
runtime: str | None = typer.Option(
None,
"--runtime",
Expand All @@ -859,6 +874,7 @@ def deploy_command(
no_env=no_env,
no_cache=no_cache,
verbose=verbose,
project=project,
build_args=build_args,
build_secrets=secrets,
runtime=runtime,
Expand All @@ -874,6 +890,7 @@ def deploy_command(
no_cache=no_cache,
verbose=verbose,
registry_id=registry_id,
project=project,
build_args=build_args,
build_secrets=secrets,
runtime=runtime,
Expand Down
Loading
Loading