From 817e9cd88ada6550f386596453fa3fd6d01238cd Mon Sep 17 00:00:00 2001 From: Raghav Chari Date: Mon, 29 Jun 2026 13:02:39 -0400 Subject: [PATCH 1/4] feat(0.2c): route per-iter PNGs through LivePulsePlotCallback (AbstractIntermediateCallback) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Retire the hand-rolled `plot_pulse`/`CairoMakie.save` per-iter PNG path (the PR #14 interim) in the bundled solve template. The live plot now flows through Piccolo's `LivePulsePlotCallback` — an `AbstractIntermediateCallback` — installed on the Ipopt solve via `IpoptOptions(intermediate_callback=…)` (the symbol DTO 0.2a wires into the Ipopt backend). This satisfies the Phase-0prime DoD: "β's live plot now flows through AbstractIntermediateCallback." - templates/solve_template.jl: - `LivePulsePlotCallback(qtraj, prob.trajectory; every=PLOT_EVERY, save_dir=".")` installed via `options=IpoptOptions(intermediate_callback=live_plot)`; emits `iter_.png` through the callback abstraction (inspector unchanged — it reads run-dir PNGs; its regex already accepts unbounded digits). - AMICODE_ITER text telemetry stays INTACT on the raw Ipopt callback (it needs the rich IPM state obj_value/inf_pr/inf_du the agnostic `(primal,iter)` contract can't carry). DTO composes the raw callback with the intermediate_callback so both fire per iter. - Drop `save_control_plot` + the end-of-solve fallback plot: LivePulsePlotCallback fires at iter 0 and every PLOT_EVERY, so a run always has ≥1 frame, and a direct plot would defeat routing every frame through the callback. - `callback_update_trajectory_factory` dropped — LivePulsePlotCallback already reconstructs the trajectory from the primal each iter. - AGENTS.md: document `LivePulsePlotCallback`/`AbstractIntermediateCallback` as the blessed per-iter plot idiom; tell the agent never to hand-roll plotting. Live inspector is ipopt-only for v1 (Q74). Repo grep confirms no plot_pulse/save_control_plot/CairoMakie.save remains in the template; template parses clean. NOTE: cannot go green end-to-end until DTO 0.2a is released (v0.9.7, the intermediate_callback field) and the bundled julia Manifest (packages/extension/julia) is bumped to resolve it — a release-coordination follow-up. Fast-lane CI is unaffected (it doesn't execute the template; the template-run validation is the slow tier). Refs #21. Co-Authored-By: Claude Opus 4.8 (1M context) --- packages/extension/AGENTS.md | 27 ++++++++++- .../extension/templates/solve_template.jl | 48 ++++++++----------- 2 files changed, 44 insertions(+), 31 deletions(-) diff --git a/packages/extension/AGENTS.md b/packages/extension/AGENTS.md index 75e41057..4a581848 100644 --- a/packages/extension/AGENTS.md +++ b/packages/extension/AGENTS.md @@ -63,8 +63,12 @@ gate, tell them plainly it isn't supported yet and stop. script, running with cwd = the run dir, must emit: - `AMICODE_ITER iter= f= inf_pr=<…> inf_du=<…>` to stdout, flushed, - once per Ipopt iteration (drives the live stats row). -- `iter_.png` every few iterations (the live plot the Inspector shows). + once per Ipopt iteration (drives the live stats row). This stays on the raw + Ipopt callback — it needs the rich IPM state the agnostic callback can't carry. +- `iter_.png` every few iterations (the live plot the Inspector shows). These + flow through **`LivePulsePlotCallback`** — Piccolo's `AbstractIntermediateCallback` + (the blessed per-iter plot idiom; see below). Do NOT hand-roll a per-iter + `plot_pulse`/`CairoMakie.save`. - `result.toml`, written **atomically** (write `result.toml.tmp`, then `mv`), with at least `fidelity` (float) and `iterations` (int). - `pulse.jld2` (the solved pulse) via `JLD2.save`. @@ -72,6 +76,25 @@ script, running with cwd = the run dir, must emit: The template already does all of this — you only fill in numbers. +### Per-iter plotting idiom (`LivePulsePlotCallback`) + +The live plot is emitted by `LivePulsePlotCallback`, which subtypes +DirectTrajOpt's solver-agnostic `AbstractIntermediateCallback`. You install it on +the solve via the solver's `intermediate_callback` option — the template wires +the Ipopt path (live inspector is ipopt-only, Q74): + +```julia +live_plot = LivePulsePlotCallback(qtraj, prob.trajectory; every = 6, save_dir = ".") +solve!(qcp; max_iter = max_iter, + options = IpoptOptions(intermediate_callback = live_plot), # → iter_.png + callback = CB.callback_factory(cb_log)) # → AMICODE_ITER text +``` + +It reconstructs the pulse from the optimizer's primal each iteration and writes +`iter_.png` into the run dir — the same callback object would install on MadNLP +via `MadNLPOptions(intermediate_callback = live_plot)`. Keep the per-iter PNGs +flowing through this callback; never re-introduce a bespoke plotting block. + ## Warm-start idiom To seed from a previous solve: `traj = load_traj("path/to/pulse.jld2")` and diff --git a/packages/extension/templates/solve_template.jl b/packages/extension/templates/solve_template.jl index c5781c3b..6d26eb33 100644 --- a/packages/extension/templates/solve_template.jl +++ b/packages/extension/templates/solve_template.jl @@ -5,7 +5,7 @@ # Vetted against Piccolo 1.19 (the version `Pkg.add Piccolo` installs today): a # single-qubit X gate on a 3-level transmon converges to subspace fidelity ~1.0. using Piccolo -using CairoMakie +using CairoMakie # loads PiccoloMakieExt → gives LivePulsePlotCallback its impl using JLD2 using TOML using Printf @@ -31,43 +31,32 @@ qcp = SmoothPulseProblem(qtraj, N; Q = 100.0, R = 1e-2) prob = hasproperty(qcp, :prob) ? qcp.prob : qcp -# Per-iter callbacks via Piccolo's PUBLIC `Callbacks` module (Piccolo 1.19). -# NOTE on solver portability: this is the Ipopt intermediate-callback path -# (rich state: obj_value/inf_pr/inf_du). When the default solver moves to -# MadNLP/Altissimo, migrate to the solver-agnostic `AbstractIntermediateCallback` -# (e.g. `LivePulsePlotCallback`), which fires `(primal, iter)` across backends. -const CB = Piccolo.Callbacks - -# Plot every 6 iters (frequent live frames), skipping iter-0. Edge case: a solve -# that converges in <6 iters emits no per-iter frame — the inspector shows -# "warming up" until the end-of-solve guarantee frame below. Acceptable: the -# warming-up state covers it, and sub-6-iter solves are rare in this regime. +# Per-iter live plot flows through Piccolo's `LivePulsePlotCallback`, an +# `AbstractIntermediateCallback` (the blessed, solver-agnostic per-iter plot +# idiom — see AGENTS.md). It reconstructs the pulse from the optimizer's primal +# each iteration and writes `iter_.png` into the run dir; the Run Inspector +# reads those frames. `every` is the redraw cadence. (No hand-rolled plotting: +# the PNGs are the callback's job, not the script's.) const PLOT_EVERY = 6 +live_plot = LivePulsePlotCallback(qtraj, prob.trajectory; every = PLOT_EVERY, save_dir = ".") + +# AMICODE_ITER text telemetry stays on the RAW Ipopt callback — it needs the rich +# IPM state (obj_value/inf_pr/inf_du) that the agnostic `(primal, iter)` contract +# doesn't carry. Both callbacks fire once per iteration (DTO composes the raw +# callback with `intermediate_callback`); the live inspector is ipopt-only (Q74). +const CB = Piccolo.Callbacks iters = Ref(0) function cb_log(optimizer, st; kwargs...) k = Int(st.iter_count); iters[] = k @printf("AMICODE_ITER iter=%d f=%.6e inf_pr=%.3e inf_du=%.3e\n", k, st.obj_value, st.inf_pr, st.inf_du) flush(stdout) - (k > 0 && k % PLOT_EVERY == 0) && save_control_plot(k) # skip iter-0 (just the random init; defers Makie's first-plot compile off the first iter) return true end -# Per-iter pulse plot via Piccolo's canonical `plot_pulse` (no rollout — keeps -# the live solve fast). `bounds=true` shades the drive bounds; the QCP method -# reads the current optimizer iterate, which callback_update_trajectory_factory -# keeps in sync. Returns a Makie Figure we save as the run-dir frame. -function save_control_plot(k::Int) - try - fig = plot_pulse(qcp; bounds = true, title = @sprintf("iter %d", k)) - CairoMakie.save(@sprintf("iter_%04d.png", k), fig) - catch e - @warn "iter plot failed" exception = e # never let plotting kill the solve - end -end - t0 = time() solve!(qcp; max_iter = max_iter, print_level = 1, - callback = CB.callback_factory(CB.callback_update_trajectory_factory(prob), cb_log)) + options = IpoptOptions(intermediate_callback = live_plot), + callback = CB.callback_factory(cb_log)) wall = time() - t0 # Fidelity over the COMPUTATIONAL subspace, from a fresh high-tolerance rollout. @@ -80,8 +69,9 @@ wall = time() - t0 Uroll = iso_vec_to_operator(unitary_rollout(get_trajectory(qcp), sys)[:, end]) fid = unitary_fidelity(Uroll, op.operator; subspace = op.subspace) -# ensure at least one PNG even if the solve stopped before PLOT_EVERY -isfile(@sprintf("iter_%04d.png", iters[])) || save_control_plot(iters[]) +# LivePulsePlotCallback fires at iter 0 (and every PLOT_EVERY after), so a run +# dir always has ≥1 frame; no end-of-solve fallback plot is needed (and a direct +# plot here would defeat routing every frame through the callback abstraction). JLD2.save("pulse.jld2", "traj", prob.trajectory) # key "traj" so `load_traj` can reload it (warm-start) open("result.toml.tmp", "w") do io From c860d28e21d9423e6e0aee3f4fe0005d43129e74 Mon Sep 17 00:00:00 2001 From: Raghav Chari Date: Tue, 30 Jun 2026 11:01:31 -0400 Subject: [PATCH 2/4] =?UTF-8?q?docs(0.2c):=20AGENTS.md=20=E2=80=94=20dual-?= =?UTF-8?q?state=20plotting=20idiom=20(prefer=20LivePulsePlotCallback=20on?= =?UTF-8?q?ce=20DTO=E2=89=A50.9.7=20pinned)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review (Jack #43): don't ban the hand-rolled Ipopt-callback PNG path — it's the only one that runs on the pinned DTO 0.9.6 and the one D10 prescribes for β. Soften the idiom guidance to a conditional: - prefer LivePulsePlotCallback once the bundled Julia project pins DirectTrajOpt ≥ 0.9.7 (the Ipopt intermediate_callback field); - otherwise hand-roll plot_pulse/CairoMakie.save from the raw Ipopt callback (IpoptOptions(intermediate_callback=…) throws at construction on 0.9.6). Note the bundled template's LivePulsePlotCallback path lands in lockstep with the DTO ≥0.9.7 Manifest bump, so template + pin never diverge on main. Co-Authored-By: Claude Opus 4.8 (1M context) --- packages/extension/AGENTS.md | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/packages/extension/AGENTS.md b/packages/extension/AGENTS.md index 4a581848..3c18e4cd 100644 --- a/packages/extension/AGENTS.md +++ b/packages/extension/AGENTS.md @@ -65,10 +65,10 @@ script, running with cwd = the run dir, must emit: - `AMICODE_ITER iter= f= inf_pr=<…> inf_du=<…>` to stdout, flushed, once per Ipopt iteration (drives the live stats row). This stays on the raw Ipopt callback — it needs the rich IPM state the agnostic callback can't carry. -- `iter_.png` every few iterations (the live plot the Inspector shows). These - flow through **`LivePulsePlotCallback`** — Piccolo's `AbstractIntermediateCallback` - (the blessed per-iter plot idiom; see below). Do NOT hand-roll a per-iter - `plot_pulse`/`CairoMakie.save`. +- `iter_.png` every few iterations (the live plot the Inspector shows). See + the per-iter plotting idiom below — **`LivePulsePlotCallback`** once the bundled + Julia project pins DirectTrajOpt ≥ 0.9.7, else the hand-rolled Ipopt-callback + path (the only one that runs on 0.9.6). - `result.toml`, written **atomically** (write `result.toml.tmp`, then `mv`), with at least `fidelity` (float) and `iterations` (int). - `pulse.jld2` (the solved pulse) via `JLD2.save`. @@ -76,12 +76,16 @@ script, running with cwd = the run dir, must emit: The template already does all of this — you only fill in numbers. -### Per-iter plotting idiom (`LivePulsePlotCallback`) +### Per-iter plotting idiom -The live plot is emitted by `LivePulsePlotCallback`, which subtypes -DirectTrajOpt's solver-agnostic `AbstractIntermediateCallback`. You install it on -the solve via the solver's `intermediate_callback` option — the template wires -the Ipopt path (live inspector is ipopt-only, Q74): +Two idioms, by what the bundled Julia project pins: + +**Preferred — once DirectTrajOpt ≥ 0.9.7 is pinned: `LivePulsePlotCallback`.** +It subtypes DirectTrajOpt's solver-agnostic `AbstractIntermediateCallback` and is +installed via the solver's `intermediate_callback` option (the Ipopt path; live +inspector is ipopt-only, Q74). It reconstructs the pulse from the optimizer's +primal each iteration and writes `iter_.png` — the same object would install +on MadNLP via `MadNLPOptions(intermediate_callback = …)`: ```julia live_plot = LivePulsePlotCallback(qtraj, prob.trajectory; every = 6, save_dir = ".") @@ -90,10 +94,16 @@ solve!(qcp; max_iter = max_iter, callback = CB.callback_factory(cb_log)) # → AMICODE_ITER text ``` -It reconstructs the pulse from the optimizer's primal each iteration and writes -`iter_.png` into the run dir — the same callback object would install on MadNLP -via `MadNLPOptions(intermediate_callback = live_plot)`. Keep the per-iter PNGs -flowing through this callback; never re-introduce a bespoke plotting block. +**Fallback on DirectTrajOpt 0.9.6 (no Ipopt `intermediate_callback` field yet): +hand-roll the PNG from the raw Ipopt callback** — `IpoptOptions(intermediate_callback=…)` +throws at construction on 0.9.6, so if you author a script against a project still +pinned to 0.9.6, use the text-callback path instead: in `cb_log`, every few iters +call `plot_pulse(qcp; bounds = true, title = …)` and `CairoMakie.save` the figure +(alongside `callback_update_trajectory_factory` to keep the iterate in sync). + +The bundled template uses the preferred `LivePulsePlotCallback` path; it lands +together with the DirectTrajOpt ≥ 0.9.7 `Manifest.toml` bump (lockstep), so the +template and the pin are never out of step on `main`. ## Warm-start idiom From 3ab9d327cf3912a715cd247b0397691d5a4ce4f5 Mon Sep 17 00:00:00 2001 From: Jack Champagne Date: Wed, 1 Jul 2026 11:02:11 -0400 Subject: [PATCH 3/4] =?UTF-8?q?chore(0.2c):=20bump=20DirectTrajOpt=200.9.6?= =?UTF-8?q?=20=E2=86=92=200.9.7=20(Ipopt=20intermediate=5Fcallback)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- packages/extension/julia/Manifest.toml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/extension/julia/Manifest.toml b/packages/extension/julia/Manifest.toml index 5baf2112..d4a47f8c 100644 --- a/packages/extension/julia/Manifest.toml +++ b/packages/extension/julia/Manifest.toml @@ -561,9 +561,9 @@ version = "0.7.18" [[deps.DirectTrajOpt]] deps = ["ExponentialAction", "FiniteDiff", "ForwardDiff", "Ipopt", "LazyArrays", "Libdl", "LinearAlgebra", "MathOptInterface", "NamedTrajectories", "OrdinaryDiffEqTsit5", "Random", "Reexport", "SciMLBase", "SparseArrays", "Test", "TestItemRunner", "TestItems", "TrajectoryIndexingUtils"] -git-tree-sha1 = "ccd269fd67bf08b4b4ae789e3c39c44de254c596" +git-tree-sha1 = "c722d301a8d064de5d9ab67e5bc8a1dff893683a" uuid = "c823fa1f-8872-4af5-b810-2b9b72bbbf56" -version = "0.9.6" +version = "0.9.7" [deps.DirectTrajOpt.extensions] MadNLPSolverExt = ["MadNLP"] From dc602eafb728227183ebfff19507c5ebe6427152 Mon Sep 17 00:00:00 2001 From: Raghav Chari Date: Wed, 1 Jul 2026 11:52:24 -0400 Subject: [PATCH 4/4] fix(0.2c): end-of-solve guarantee frame via LivePulsePlotCallback (review #43) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Jack's re-review item (2): the migration dropped the end-of-solve frame, so a solve that converges in < PLOT_EVERY iters would leave only the iter-0 random-init frame. Restore the guarantee — but keep it flowing through LivePulsePlotCallback (no bespoke plot_pulse): re-invoke the callback once at every=1 with the FINAL primal (reconstructed from prob.trajectory, which DTO syncs to the final iterate after solve!). Short solves now end on the converged pulse; typical solves overwrite the last multiple-of-PLOT_EVERY frame with the same content. Co-Authored-By: Claude Opus 4.8 (1M context) --- packages/extension/templates/solve_template.jl | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/packages/extension/templates/solve_template.jl b/packages/extension/templates/solve_template.jl index 6d26eb33..0f93d012 100644 --- a/packages/extension/templates/solve_template.jl +++ b/packages/extension/templates/solve_template.jl @@ -69,9 +69,18 @@ wall = time() - t0 Uroll = iso_vec_to_operator(unitary_rollout(get_trajectory(qcp), sys)[:, end]) fid = unitary_fidelity(Uroll, op.operator; subspace = op.subspace) -# LivePulsePlotCallback fires at iter 0 (and every PLOT_EVERY after), so a run -# dir always has ≥1 frame; no end-of-solve fallback plot is needed (and a direct -# plot here would defeat routing every frame through the callback abstraction). +# End-of-solve guarantee frame — STILL through LivePulsePlotCallback (no bespoke +# plot). The live callback fires at iters 0, PLOT_EVERY, 2·PLOT_EVERY, …; a solve +# that converges in < PLOT_EVERY iters would otherwise leave only the iter-0 +# random-init frame (inspector stuck showing the initial guess). Re-invoke the +# callback once at every=1 with the FINAL primal so the last frame is the +# converged pulse. prob.trajectory is the final iterate here (DTO synced it after +# solve!), so this reconstructs the same primal the callback saw per-iter. +let final_cb = LivePulsePlotCallback(qtraj, prob.trajectory; every = 1, save_dir = ".") + tr = prob.trajectory + final_primal = tr.global_dim > 0 ? vcat(collect(tr.datavec), collect(tr.global_data)) : collect(tr.datavec) + final_cb(final_primal, iters[]) +end JLD2.save("pulse.jld2", "traj", prob.trajectory) # key "traj" so `load_traj` can reload it (warm-start) open("result.toml.tmp", "w") do io