feat(mpc): run compiled Energyplan first in beta with Core DP shadow - #1077
Conversation
Go fallback used to skip ValidatePlan, and forward simulation snapped SoC onto the DP grid, so reported energy did not replay. Python serialized full curtailment as pv_limit_w=0, the same sentinel as no cap, and applied the PV-charge bonus in every mode. Share BatteryEnergyDeltaWh, replay aggregate Go DP trajectories, start forward sim from real SoC, and add pv_curtail_active so a zero cap is a distinct contract. Gate the PV-charge bonus to passive_arbitrage. Leftover-PV and battery-to-EV residual constraints run per scenario. Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Policy lookup stays on the SoC grid. Forward simulation now integrates from the actual initial SoC, including starts outside the configured band, so ValidatePlan and the published trajectory match. An action that would worsen recovery is clipped to the remaining headroom instead of being replaced with idle or snapped after the fact. Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
frahlg
changed the title
912d4ce to
0ad03bb
Compare
0ad03bb to
e005853
Compare
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
frahlg
changed the title
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 761dd6b. Configure here.
| "energyplan_minus_core_ore_terminal_corrected", block.ActiveMinusShadowTerminalCorrectedOre, | ||
| "core_solve_ms", block.Solver.SolveMs) | ||
| } | ||
| s.recordCoreDPShadow(champion, slots, p, reason, replanAtMs, block) |
There was a problem hiding this comment.
Cancelled shadow stored as rejected
Medium Severity
A new replan cancels the in-flight Core DP shadow, but context.Canceled is treated like a failed solve and written onto the still-current plan as rejected. The busy flag also stays set until that goroutine exits, so startCoreDPShadow can skip the replacement comparison. The UI then shows a false unavailable verdict and may miss the next one.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 761dd6b. Configure here.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 761dd6b099
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // plan itself scheduled. Consumed by the control loop only when | ||
| // the driver advertises `supports_pv_curtail`. | ||
| PVLimitW float64 `json:"pv_limit_w,omitempty"` | ||
| PVCurtailActive bool `json:"pv_curtail_active,omitempty"` |
There was a problem hiding this comment.
When Energyplan returns pv_curtail_active=true with pv_limit_w=0—the full-curtail case—the flag exists only on Action: Service.SlotDirective copies only PVLimitW, and control dispatch activates planner curtailment only when that value is greater than zero. The accepted plan therefore models zero PV but sends no curtail command, causing negative-price or costly-export slots to execute against a different power balance. Carry the active flag through the MPC-to-control bridge or otherwise encode a dispatchable zero cap.
Useful? React with 👍 / 👎.
| if mode != "passive_arbitrage": | ||
| return 0.0 |
There was a problem hiding this comment.
For pv_charge_bonus_ore_kwh > 0, this new mode gate makes the Python optimizer ignore the bonus in self-consumption, cheap-charge, and arbitrage, while Core's Optimize still subtracts it in every mode at go/internal/mpc/mpc.go:943-951. Consequently the default Core/Python shadow comparison and an explicit Python champion solve different objectives for those modes, so comparison deltas no longer measure planner quality and the engines can select different actions solely because of the mismatch. Either apply the same gate in Core or retain the bonus in Python for all modes.
Useful? React with 👍 / 👎.
| # platform-specific syscall/import mistakes before push. | ||
|
|
||
| verify: test compose-migration-test container-boundary-test release-workflow-test | ||
| verify: test compose-migration-test container-boundary-test release-workflow-test native-solver-test |
There was a problem hiding this comment.
Making native-solver-test an unconditional dependency means make verify now always runs verify.py, whose host_key() rejects any host outside Linux amd64/arm64 and Darwin arm64 before the integrity-only checks can finish; Darwin amd64, Windows, and other development hosts therefore cannot run the project's standard verification even though the application deliberately falls back to Core there. Keep bundle checksum/unit checks universal, but conditionally skip the host handshake and native Go integration tests when no bundled executable matches the host.
AGENTS.md reference: AGENTS.md:L114-L115
Useful? React with 👍 / 👎.
1 participant
Beta releases now start the compiled Energyplan worker when
planner.engineis unset on a supported host. Core validates its plan before publishing it, then runs a bounded Core DP shadow on the same downside PV forecast. Explicitcoreandpythonsettings keep their roles; stable and development builds keep Core as the unset default.The bundle contains Linux amd64/arm64 and macOS arm64 workers, a pinned checksum manifest, license and notices. Docker images and release archives carry the bundle. No Rust source, source tests or build tools enter this public repository; those remain in private
srcfl/energyplan. The worker updates with Core, and the UI removes its separate sidecar update control.The worker has a 500 ms solve budget and a 2 s process timeout. Core DP shadow runs after publication, with one comparison at a time and a 10 s deadline. A new replan cancels the old comparison. Results attach only to the matching decision ID. Both plans use Core's grid cost model; the comparison also reports the value of energy left at the end. Rejected shadows show no cost verdict.
This branch includes #971's replay fixes and its already-merged #957 dependency. Core validates fallback plans, DP clips power at the battery limits, and the validator rejects EV energy above capacity. When Energyplan cannot model operating-band recovery, Core can plan from the real initial SoC and must pass replay. Unsupported multiple assets and other models return an error; fallback remains subject to validation. A failed replay leaves the prior plan in place. Driver dispatch and its safety guards remain authoritative.
Validation:
make verify: Go/Python tests, contract and release checks, vet and build pass.Overlap: #971 is included as an ancestor. #1036 retains the Python-shadow cost review and its replay benchmarks; this change adds the Energyplan-to-Core comparison separately. #963's peak-flattening policy stays outside this change.
The repository owner's beta request authorizes release once CI passes. Human field review and stable promotion remain separate.
Note
High Risk
Changes the default planner on beta releases and adds mandatory plan validation on the dispatch path—mistakes could block replans or alter live energy schedules.
Overview
Beta sites without an explicit
planner.enginenow use the bundled Energyplan (ftw-solver) worker on supported hosts (not Windows). Core validates the worker plan, publishes it, then runs a bounded Core DP shadow on the same downside-PV slots for measurement only. Worker failure, timeout, or failed replay falls back to Core DP with visible solver metadata; invalid fallback keeps the previous plan. Explicitcore,python, andenergyplanstill select those engines; stable/dev unset defaults stay on Core. The proprietary binaries, manifest, and licenses ship in Docker images and release archives, with CI viamake native-solver-test/native-solver.yml. Bundled optimizers skip the separate Python sidecar updater; the components API exposesbundled_with_core.Planning safety and parity tighten across engines. Every champion plan (including Go DP fallback) must pass
ValidatePlanphysical replay before dispatch. PV curtailment addspv_curtail_activeso a zero cap is not confused with “no limit”; Python and Go paths share the encoding. Validation accepts sub-watt fuse residue from external solvers but still rejects real limit breaches; EV energy above capacity is rejected; Core DP clips power near SoC band edges and integrates energy from the real initial SoC. Shared helpers (BatteryEnergyDeltaWh,EffectivePVW) align replay with site physics.Reviewed by Cursor Bugbot for commit 761dd6b. Bugbot is set up for automated code reviews on this repo. Configure here.