Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion change_log.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
## Unreleased

### Changed
* Experimental `-compile-hfir-bc` compiles and runs `defun`, `call`, and `return` with the existing `CALL` and `RETURN` opcodes. `while` is still rejected on that path. Production `-compile-bc` is still AST bytecode after the gate. #90 stays Partial. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase3a_defun_call.md`.
* Experimental `-compile-hfir-bc` compiles and runs `while` with the existing `JUMP_IF_FALSE` and `JUMP` opcodes. A while header's control edges are the condition and then the body. Production `-compile-bc` is still AST bytecode after the gate. #90 stays Partial. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase3b_while.md`.
* Experimental `-compile-hfir-bc` compiles and runs `defun`, `call`, and `return` with the existing `CALL` and `RETURN` opcodes. Production `-compile-bc` is still AST bytecode after the gate. #90 stays Partial. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase3a_defun_call.md`.
* Generated Go and JavaScript mediate `(fetch)`. An empty, missing, or non-`network` `HOWLFRAME_ALLOW_CAPS` grant fails with `CAPABILITY_DENIED` before any HTTP request, and the denial does not include the URL. The `network` grant performs the request. The production `-compile-bc` path is unchanged. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase2d_fetch.md`.
* Generated Go and JavaScript mediate `(read_file)`. An empty, missing, or non-`filesystem` `HOWLFRAME_ALLOW_CAPS` grant fails with `CAPABILITY_DENIED` before any filesystem read, and the denial does not include the path. The `filesystem` grant reads the file. The production `-compile-bc` path is unchanged. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase2c_read_file.md`.
* Generated Go and JavaScript mediate `(exec)`. An empty, missing, or non-`process` `HOWLFRAME_ALLOW_CAPS` grant fails with `CAPABILITY_DENIED` before a subprocess starts, and the denial does not include the command. The `process` grant runs it. The production `-compile-bc` path is unchanged. Journal: `docs/journals/2026-09-30_lowered_hfir_abi_phase2b_exec.md`.
Expand Down
13 changes: 7 additions & 6 deletions docs/hfir_execution_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Semantic information added for this path includes literal kind, explicit program

## What AST still owns

The parser, source expansion, module resolution, patch/context transformations, checker rules, construct-position classification, and public build integration still operate on the AST. The legacy AST bytecode compiler remains the production compiler. Phase 3a teaches the experimental lowerer `defun`, `call`, and `return` with the existing `CALL` and `RETURN` opcodes. `while` and the rest of the control-frame layout stay on the AST compiler.
The parser, source expansion, module resolution, patch/context transformations, checker rules, construct-position classification, and public build integration still operate on the AST. The legacy AST bytecode compiler remains the production compiler. Phase 3a teaches the experimental lowerer `defun`, `call`, and `return` with the existing `CALL` and `RETURN` opcodes. Phase 3b teaches it `while` with the existing `JUMP_IF_FALSE` and `JUMP` opcodes. The rest of the control-frame layout stays on the AST compiler.

## Phase-1 executable subset

Expand All @@ -39,13 +39,14 @@ The direct lowerer supports a deterministic `cli_app` subset:
| Deterministic mutation | `map_set`, `map_delete`, `append` |
| Observability | `print`, `stderr`, `exit` |
| Capability evidence | `env`, using the existing shared capability authority |
| Functions (Phase 3a) | `defun`, `param`, `call`, `return`. Existing `CALL` and `RETURN` opcodes. `while` is not included. |
| Functions (Phase 3a) | `defun`, `param`, `call`, `return`. Existing `CALL` and `RETURN` opcodes. |
| Loops (Phase 3b) | `while`. Control edges are the condition, then the body. Existing `JUMP_IF_FALSE` and `JUMP` opcodes. |

## Unsupported HFIR nodes

Every node outside the subset fails closed with one `HFIR_BYTECODE_UNSUPPORTED` error diagnostic. The diagnostic identifies the offending graph node, target `bytecode`, and available source provenance. It returns no `BCProgram`.

Phase 3a moved `defun`, `call`, and `return` into the experimental subset. `while` stays deferred: `LowerToBytecode` still returns `HFIR_BYTECODE_UNSUPPORTED` and no `BCProgram`. Other examples that remain outside a full CFG include loops the lowerer has not been asked to treat as SSA, `try_let` and `catch` where the graph is still not the production source, HTTP routes and lambdas, stores, and model-oriented operations. `ControlEdges` are still empty. This is not a claim that `while` cannot be represented later.
Phase 3a moved `defun`, `call`, and `return` into the experimental subset. Phase 3b moves `while` in as well: `LowerAST` fills that node's `ControlEdges` with the condition and then the body, and `LowerToBytecode` emits the existing jumps. A `while` without those edges still returns `HFIR_BYTECODE_UNSUPPORTED` and no `BCProgram`. Other nodes still have empty `ControlEdges`. Examples that remain outside a full CFG include `for` treated as SSA, `try_let` and `catch` where the graph is still not the production source, HTTP routes and lambdas, stores, and model-oriented operations. This is not a claim that the graph is SSA.

## Bytecode ownership

Expand All @@ -72,19 +73,19 @@ HowlChangeOps needs 23 runtime constructs plus `catch`; Phase 1 does not support

## HowlBoard compatibility

The existing HowlBoard backend compatibility suite passes against the baseline HowlFrame bytecode compiler, including HTTP request parsing, JSON dict/list behavior, stores, CORS, and network/database capability behavior. Its browser test is blocked locally only because Playwright Chromium is not installed. HowlBoard requires routes and lambdas, HTTP response forms, request parsing, stores, and loops, so it remains outside this experimental subset. Phase 3a covers `defun`, `call`, and `return` on `-compile-hfir-bc` only. The production path is unchanged.
The existing HowlBoard backend compatibility suite passes against the baseline HowlFrame bytecode compiler, including HTTP request parsing, JSON dict/list behavior, stores, CORS, and network/database capability behavior. Its browser test is blocked locally only because Playwright Chromium is not installed. HowlBoard requires routes and lambdas, HTTP response forms, request parsing, stores, and loops, so it remains outside this experimental subset. Phase 3a covers `defun`, `call`, and `return` on `-compile-hfir-bc` only. Phase 3b covers `while` on that same flag. The production path is unchanged.

## What must happen before #88

Improvement #88 has a real but deliberately bounded execution destination: model-authored graphs that meet the Phase-1 schema can be verified and lowered directly to a deterministic artifact, while unsupported nodes fail closed. Phase 3a adds source-level `defun`, `call`, and `return` on `-compile-hfir-bc`. The model-adapter transport still rejects those kinds, so this slice does not reopen #88. Structured error recovery, `while`, and control-flow edges remain. Work on #88 is still constrained Phase-1 adapter design, not a claim that arbitrary model-authored HFIR can execute today.
Improvement #88 has a real but deliberately bounded execution destination: model-authored graphs that meet the Phase-1 schema can be verified and lowered directly to a deterministic artifact, while unsupported nodes fail closed. Phase 3a adds source-level `defun`, `call`, and `return` on `-compile-hfir-bc`. Phase 3b adds source-level `while` and that loop's control edges on the same flag. The model-adapter transport still rejects those kinds, so this slice does not reopen #88. Structured error recovery and a real CFG remain. Work on #88 is still constrained Phase-1 adapter design, not a claim that arbitrary model-authored HFIR can execute today.

## Lowered ABI v1 (improvement #90, phase 1)

`docs/reference/lowered_hfir_abi_v1.md` versions the contract the hosts must share (`lowered-hfir-abi/v1`). The conformance suite compares the interpreter, the bytecode VM, Go, and JavaScript on a pure core and on `env` denial. That comparison runs the production AST paths through `tools/difftest`. It does not switch `-compile-bc` over to `LowerToBytecode`.

Phase 2a mediates `env` in generated Go and JavaScript. An empty `HOWLFRAME_ALLOW_CAPS` grant is `CAPABILITY_DENIED` and does not read the variable. Phase 2b mediates `exec` the same way: an empty grant, or a grant that omits `process`, is `CAPABILITY_DENIED` before any subprocess starts. Phase 2c mediates `read_file` the same way: an empty grant, or a grant that omits `filesystem`, is `CAPABILITY_DENIED` before any filesystem read. Phase 2d mediates `fetch` the same way: an empty grant, or a grant that omits `network`, is `CAPABILITY_DENIED` before any HTTP request. The production compiler is unchanged.

Wasm feasibility in this revision is the closed set `exec`, `spawn_agent`, and `http_server_start` (`HFIR_TARGET_INFEASIBLE`). Control edges are still unpopulated. Phase 3a makes `defun`, `call`, and `return` executable on `-compile-hfir-bc` only. `while` is still not executable HFIR. Production `-compile-bc` is still the AST. The rest of Phase 2 is one lowered graph for every host. Journals: `docs/journals/2026-09-30_lowered_hfir_abi_phase1.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2a_env.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2b_exec.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2c_read_file.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2d_fetch.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase3a_defun_call.md`.
Wasm feasibility in this revision is the closed set `exec`, `spawn_agent`, and `http_server_start` (`HFIR_TARGET_INFEASIBLE`). Phase 3a makes `defun`, `call`, and `return` executable on `-compile-hfir-bc` only. Phase 3b makes `while` executable on that flag and fills `ControlEdges` for the loop header only. Production `-compile-bc` is still the AST. The rest of Phase 2 is one lowered graph for every host. Journals: `docs/journals/2026-09-30_lowered_hfir_abi_phase1.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2a_env.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2b_exec.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2c_read_file.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase2d_fetch.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase3a_defun_call.md`, `docs/journals/2026-09-30_lowered_hfir_abi_phase3b_while.md`.

## Provenance limitation

Expand Down
14 changes: 8 additions & 6 deletions docs/hfir_failure_localization_status.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ and cannot be localized from that map.

# Control-flow representation

The existing persisted `ControlEdges` field is not used: it has no role,
direction, ordering, lowering, or model-transport semantics. Phase 3 instead
derives a read-only Phase-1 control view from validated canonical roles:
program and sequence body order, let body continuation, and if then or else
branch containment. No model-supplied control edge is accepted. `exit` remains
terminal runtime behavior rather than an arbitrary graph edge.
Phase 3b writes ordered control successors on a source-lowered `while`
header: the condition, then the body. The experimental lowerer follows that
pair. Localization does not. It still derives a read-only Phase-1 control
view from validated canonical roles: program and sequence body order, let
body continuation, if then or else branch containment, and a `while` only
when those persisted successors match the condition and body roles. No
model-supplied control edge is accepted. `exit` remains terminal runtime
behavior rather than an arbitrary graph edge.

# Runtime trace

Expand Down
38 changes: 38 additions & 0 deletions docs/journals/2026-09-30_lowered_hfir_abi_phase3b_while.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Lowered HFIR ABI, phase 3b: while and control edges on the experimental bytecode path

## Why this slice

Phase 3a made `defun`, `call`, and `return` executable on `-compile-hfir-bc`. `while` still failed closed with `HFIR_BYTECODE_UNSUPPORTED`, and `LowerAST` still left `ControlEdges` empty. The production hosts already run `while`. This slice makes the experimental path run that loop, and records the two control successors the jump layout follows.

This is not a CFG or SSA pass. #90 stays Partial.

## What landed

`LowerAST` gives `(while cond body)` a condition data edge, a body data edge, and `ControlEdges` in that order: the test, then the body. The back edge is the existing `JUMP` back to the test. `if`, `for`, `defun`, and every other node stay without control edges.

`LowerToBytecode` follows those control edges and emits the existing `JUMP_IF_FALSE` and `JUMP` opcodes, with the same relative offsets as the AST bytecode compiler. A `while` whose control edges are missing, swapped, or not the condition and the body returns one `HFIR_BYTECODE_UNSUPPORTED` diagnostic and no `BCProgram`. There is no new opcode and no new capability. A `defun` body may contain `while`. The model-adapter transport still rejects kind `while`, so #88 stays closed. Localization still does not treat a model-supplied control edge as authority.

`tests/conformance/abi_v1/10_while.howl` is the shared fixture. `tools/difftest` runs it as `while_control`. The false loop must not print. The counting loop prints `1`, `2`, and `3`.

| Host | How this case reaches it |
| --- | --- |
| `hfir_bytecode` | `-compile-hfir-bc`, then `-run-bc`. Experimental path. |
| `bytecode` | `-compile-bc`, then `-run-bc`. Production AST bytecode. Canonical result. |
| `interpreter`, `go`, `javascript` | Still the AST. The fixture is already in their subset, so they are included. JavaScript rewrites the root to `web_app`. |

`internal/vm/hfir_equivalence_test.go` also compares the experimental artifact with AST bytecode on a counting loop, a loop that does not enter, a nested loop, a loop inside `defun`, and a non-bool condition. The non-bool condition is `TYPE_ERROR` on both emitters, with empty stdout. The checker rejects that program before either compiler on the CLI; the comparison is the two bytecode paths.

Production `-compile-bc` is still `runHFIRGate` and then `bytecode.CompileToBytecode` on the AST. `sleep` still compiles on that flag and `-compile-hfir-bc` still rejects it without writing an artifact.

## What is still open

* `-compile-bc` still compiles the AST. One lowered graph for every host is still the rest of Phase 2.
* Control edges exist on `while` headers only. The graph is not SSA, and the verifier still does not check cycles.
* `if` and `for` do not grow control edges in this slice.
* Generated `write_file`, `mkdir`, and the other host effects that Phase 2 has not mediated still do not consult a grant. `env`, `exec`, `read_file`, and `fetch` do.
* Feasibility is still a Wasm-only set. This slice adds no Wasm opcode and does not execute Wasm.
* No module linker, no HFIR module graph, and no VM module opcode.

## What this does not do

No new opcode. No new capability. #102–#105 and #108 stay Done and are not reopened. #88 stays closed. #90 stays Partial. `write_file` and `mkdir` stay deferred.
Loading
Loading