IOG future planned dispatches treated as cheap home rate before EV charging earns the slot

[PlainSeer]

Summary

With native Intelligent Octopus charging enabled, PredBat appears to treat a future planned daytime IOG dispatch as a guaranteed cheap home-electricity period before any EV charging has actually occurred.

On the current Intelligent Octopus Go tariff behaviour, this is unsafe for home-battery planning because daytime dispatches are provisional and can be moved or removed by Octopus. The cheap home rate is only actually earned when the EV charges in the relevant half-hour.

I reproduced a case where PredBat could see a future daytime planned dispatch, marked that future half-hour cheap, and altered the battery plan around it. Octopus then moved the planned session before it occurred. This means PredBat can potentially force-export the home battery in anticipation of a cheap recharge opportunity that subsequently disappears.

Reproduction

Setup:

• Intelligent Octopus Go
• EV charging controlled by Octopus through a Hypervolt charger
• PredBat native Intelligent Charging enabled with switch.predbat_octopus_intelligent_charging
• PredBat has access to the Intelligent Octopus planned dispatch entity and EV SoC/charge requirement

Observed sequence:

1. Octopus creates a future daytime planned dispatch.
2. The EV is not charging yet and has not charged during that future half-hour.
3. PredBat imports the future planned dispatch and treats that half-hour as cheap for the house.
4. PredBat incorporates that assumed cheap slot into the battery optimisation. In my test this made an earlier force-export strategy possible because PredBat expected a cheap recharge opportunity later.
5. Before the planned dispatch starts, Octopus changes/moves the dispatch.
6. The previously assumed cheap home slot therefore never occurs.

This creates a real risk that the house battery is exported based on a cheap rate that was only provisional.

Why this matters with the current IOG tariff

Under the newer IOG 4-rate behaviour, a daytime planned EV dispatch is not necessarily a guaranteed cheap half-hour for the home merely because it appears in planned_dispatches.

The home daytime cheap rate is effectively conditional on the EV actually charging in that half-hour within the Intelligent charging allowance. Octopus can move planned dispatches before they occur.

So there are two different concepts that PredBat currently appears to treat as equivalent:

• planned/provisional dispatch: useful for forecasting EV charging, but can move/disappear;
• earned cheap home half-hour: safe to use as an actual cheap import rate once EV charging has genuinely occurred for that billing period.

For home-battery optimisation, relying on the first as though it were the second can result in an incorrect export/charge decision.

Expected behaviour

Ideally PredBat should distinguish between forecast EV charging slots and confirmed/earned cheap home-rate slots.

Possible approaches:

1. Future planned Intelligent dispatches may still be used to forecast EV demand, but should not automatically override the home's future import tariff for battery arbitrage until the slot becomes sufficiently confirmed/active.
2. Once EV charging actually starts during a daytime dispatch, the corresponding billing half-hour can be promoted to a confirmed cheap-home slot.
3. If a planned dispatch moves before it starts, PredBat should not have made irreversible battery decisions (particularly force export) based on the old provisional cheap slot.
4. A configurable option could allow users to choose whether planned IOG slots are trusted for home-rate optimisation, while preserving current behaviour for users on tariff arrangements where the planned slot really is guaranteed.

Possible solution

A possible approach would be to separate future EV dispatch planning from home battery tariff planning.

PredBat could continue using future planned_dispatches to forecast when the EV is likely to charge, but it should not automatically treat those future daytime dispatches as guaranteed cheap import periods for the house.

For Intelligent Octopus Go daytime dispatches outside the fixed 23:30–05:30 cheap window:

• future/unstarted dispatches: use for EV demand forecasting only, while keeping the normal home import rate for battery optimisation;
• active/confirmed dispatches: promote the relevant 30-minute settlement period to the cheap home rate and immediately re-plan;
• completed dispatches: retain as confirmed cheap periods historically.

This does mean PredBat cannot fully optimise the home battery around future daytime Intelligent slots, because Octopus may still move or revoke them. However, that seems safer than allowing PredBat to make irreversible decisions such as force-exporting or deferring charging based on a cheap rate that is not yet guaranteed.

This would complement #4483 rather than replace it: #4483 limits future slots the EV no longer needs, whereas this would address future slots the EV does still need but which remain provisional until they actually occur.

Related issues

• Use actual EV SoC to limit provisional Intelligent Octopus slots used in the Predbat plan #4482 — PredBat relying on provisional Intelligent Octopus slots that later disappear once the real EV energy requirement is satisfied. This issue is related but slightly different: it concerns treating the future provisional dispatch itself as a guaranteed cheap home tariff, which can influence home-battery export decisions before the EV has charged at all.
• Forthcoming changes to octopus intelligent go #4248 — forthcoming/current Intelligent Octopus Go 4-rate tariff behaviour, including the daytime bonus home rate tied to EV charging.

Debug file

I can provide a predbat_debug.yaml from a reproduction if useful.

[chalfontchubby]

I did have a patch in #4270 to try and prevent the house charging until the car charge was committed. I set that back to draft because it had been sitting there a long time and started trying an automation - but it does not address the planning part of the equation - only the ability to start charging.

All of this I think needs a decent sensor for car_charging_now which was a sticking point. Doing any of this speculativly seems dangerous unless we know the slot is confirmed - confirming it from the octopus intergration would be good if perhaps slower, but I can't yet see how I could do that.

[gcoan]

it is a mess. Friends of mine on IOG are just using predbat led charging because Octopus moves the slots around so much you can't plan on them, and predbat is forever in catchup mode

[chalfontchubby]

We could have a "percentage value" 0.0 to 1.0 (which is not a percentage at all but matches the other usage in the code base) reflecting how much to trust future unconfirmed slots. That might be complicated though (what does a 50% trusted future slot look like) - so a switch defaulting to off for "trust future dynamic IOG slots" so we don't plan around them by default. Predbat opportunistically charges when a slot is confirmed if it wants to.

Any idea how a middle ground of "there is a 10% chance that slot is real" would work if we went that way?

[gcoan]

I'm not on IOG so can only comment on what friends who are on IOG tell me, but I understand that Octopus can and does change the slots all the time, even up to minutes beforehand.

It would seem to me that predbat can't rely on these cheap rate slots during the day and shouldn't plan any export or import until (1) the slot has started and (2 ?) the car has started charging.

Maybe there needs to be a switch that controls this behaviour, that defaults to on, giving people the option to turn it off and revert to trusting the IOG slots as being true for planning purposes.

[PianSom]

A couple of points here:

1. car charging start is randomly a few minutes after an IOG slot begins (grid mandated, to avoid everyone turning on at once)
2. a slot can start mid-half hour, if the car is plugged in and a slot is assigned immediately. In this case the cheap rate is applied retrospectively
3. I find that - frequently - the car charge slips in to the next half hour slot by a few minutes. This is perhaps because with my Zappi Octopus operates it by assigning an energy quantum which, in my case, the Zappi fails to deliver within the half hour.
4. As Geoffrey says, Octopus frequently change plans just (usually 2 mins) before the slot is due to start. Because of the way that bottlecapdave's Octopus integration works, there is a local automation to check the API, so this is not always spotted in time by HA (and therefore Predbat). See here

Sorry - rather a stream of consciousness set of points!

FYI I have a little dashboard which shows the attributes of the _intelligent_dispatching binary_sensor, show below. When I next plug the car in I can share these if it helps. They are not always helpful - IME there is no obvious relationship between "planned" and "started" eg a planned slot which starts may take some time to move to started, and may, in fact, just go straight to completed instead etc

PS My apps.yaml includes

  watch_list:
  - '{octopus_intelligent_slot}'
  - '{octopus_ready_time}'
  - '{octopus_charge_limit}'
  - '{car_charging_planned}'
  - '{octopus_intelligent_dispatching_updated}'
  - '{car_charge_control}'

where they point to the obvious HA sensors

[chalfontchubby]

I think "stream of consciousness" is about where we are trying to deal with this.
I'm starting with a way to visualize how the slots change over time - then worry bout fixing it.

Not trusting them at all seems best bet and low hanging fruit.
Next stage trust them if they have started and the car is charging.

[PianSom]

Bottlecapdave's write-up, in case you hadn't seen - https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy/setup/account/#manually-refresh-intelligent-dispatches

[PlainSeer]

I think the switch approach is probably the safest and simplest solution.

Something like trust_future_dynamic_iog_slots, defaulting to Off, would avoid trying to assign a probability or weighting to a slot which is fundamentally binary from a billing point of view.

With the switch Off:

• future daytime planned_dispatches could still be retained for EV load forecasting;
• they would not be inserted into the home rate_import as guaranteed cheap periods;
• PredBat therefore would not force-export or defer house-battery charging in reliance on them;
• once the current half-hour is sufficiently confirmed, PredBat can promote that settlement period to cheap and immediately re-plan;
• the fixed 23:30–05:30 IOG window remains unaffected and can still be planned against normally.

I am less convinced by a 0.0–1.0 confidence value. A "50% cheap" slot does not map particularly cleanly onto the real tariff: either the half-hour ultimately qualifies for the cheap home rate or it does not. Modelling an expected price may also still encourage an export decision based on a slot that later disappears.

The harder part seems to be defining "confirmed".

From the comments above, relying purely on started_dispatches may not be robust because the Octopus integration can lag, a planned slot can jump directly to completed, charging can start a few minutes after the nominal slot start, and slots can start mid-half-hour.

I currently have a Home Assistant safeguard which effectively proves this model can work in practice. Future daytime IOG slots are not exposed to PredBat as cheap in advance. Instead, once the EV has genuinely been drawing charging power continuously for a short qualification period, the relevant 30-minute settlement period is promoted to cheap and PredBat is immediately recalculated.

That gives two useful properties:

• PredBat never exports or defers charging in reliance on a future provisional daytime slot;
• once real EV charging establishes that the current settlement period is genuinely being used, PredBat can still take advantage of the cheap home rate for the remainder of that half-hour.

The exact qualification method would ideally be generic inside PredBat rather than charger-specific. car_charging_now, where available, seems the obvious input. A configurable debounce/qualification time could avoid reacting to brief charger state changes or delayed starts.

So perhaps a native implementation could be:

1. Future daytime dispatches remain EV forecast information only.
2. The current settlement period is initially treated at the normal home rate.
3. When car_charging_now remains true for a configurable qualification period, promote the whole current 30-minute settlement period to the Intelligent cheap rate.
4. Trigger an immediate PredBat re-plan.
5. Preserve completed/confirmed periods as cheap historically.
6. Leave 23:30–05:30 unchanged as guaranteed cheap.

That would lose some forward optimisation, but I think that is preferable to making an irreversible battery decision against a future cheap rate which Octopus is still free to move or revoke.

It would also mean the behaviour I am currently achieving externally could potentially be incorporated directly into PredBat in a generic way.

[browellm]

Agree with the planned approach outlined above. It's been interesting to see the evolution of IOG over the 5 years I've been using it. A schedule would occasionally see one, and rarely two, changes over a typical plug-in period of say 18 hours. As mentioned, Octopus have moved to an exceptionally dynamic scheduler with often a couple of minutes notice which Predbat is struggling to deal with. Going with the approach of not counting these ephemeral slots for planning is the safest option.

[chalfontchubby]

Would anyone be prepared to test a patch before it's merged - I've done that previously copying python files manually onto the HA box but other approaches are possible I'm sure.
I'm on IOG with the car being controlled rather than the charger, and my charger is wired up so the load is not visible to the inverter - there are a few different configurations that it would be good to cover.

[chalfontchubby]

Opened a draft PR with an implementation of the approach discussed above: #4528

Summary, matching what @PlainSeer laid out almost exactly:

• New select.predbat_trust_future_dynamic_iog_slots (expert mode), three levels instead of a plain on/off, since "wait for completed" and "promote on car_charging_now" turned out to be genuinely different trade-offs worth choosing between rather than one replacing the other:
  • none (default) - only the fixed 23:30-05:30 window is trusted as cheap for house battery planning; every daytime dispatch stays at the normal rate until it's actually confirmed.
  • completed - trusts a daytime dispatch once Octopus reports it as completed (metered) - the safest option, at the cost of never getting the current half-hour's benefit.
  • started - also promotes the current 30-minute settlement period to cheap when car_charging_now confirms the car is genuinely drawing power, then triggers on the normal 5-minute cycle rather than waiting for the next completed_dispatches update. This is the "current settlement period promoted + immediate benefit" behaviour from the comments above.
• Trust is source-based (a _confirmed flag threaded from completed_dispatches/car_charging_now), not clock-time-based - a slot's start time passing on its own is deliberately not treated as confirmation, since Octopus can revoke a dispatch that's technically started but where the car never actually drew power.
• Future daytime slots are still used in full for EV charging forecasting regardless of this setting - only whether they're trusted as cheap for the house battery is affected.
• Also added a diagnostic: a per-car dispatch timeline logged once per 30-minute boundary (visible in predbat.log/debug.yaml), so a slot getting rescinded is visible directly in the log rather than inferred.

Not claiming this is the final word, especially the started level and its reliance on car_charging_now - genuinely looking for input given the range of configurations people here are running (car-led vs charger-led control, load visible to the inverter or not, etc).

If anyone's willing to dogfood the branch ahead of merge, that would help a lot - happy to talk through how to get the patched files onto your HA box.

[PianSom]

Nice work.

On started, why not use binary_sensor.octopus_energy_XXXXXXXXXX_intelligent_dispatching?

I've found this to be reliable. The car may well not start charging immediately when this turns on, as previously noted, but surely it is definitive that we are in a cheap slot if it is TRUE (or has been TRUE in the current slot)?

If it is FALSE and the car is charging we may not be in a cheap slot. For example, in an over-run of a previous slot, as I noted in my (3) above.

[chalfontchubby]

I'll double check, but I thought I'd seen that go high in a slot in which the car didn't charge.

[chalfontchubby]

@PianSom Thanks for pushing on this - worth being precise about what that sensor actually tells you, so we went and read the BottlecapDave Octopus Energy integration's own source rather than going on behaviour alone.

binary_sensor...intelligent_dispatching's on/off state is controlled by the integration's own intelligent_rate_mode option (intelligent/dispatching.py → get_applicable_dispatch_periods() in intelligent/__init__.py), which has two settings:

• pending_and_started_dispatches - the default (confirmed as the fallback everywhere the integration reads this setting). The sensor turns on the moment now falls inside a planned dispatch window - started_dispatches doesn't need to have anything in it at all.
• started_dispatches_only - opt-in, most people won't have touched it. Only counts genuine started_dispatches entries.

So under the default mode, "sensor is on" and "there's a planned dispatch scheduled right now" are the same fact, from the same data Predbat's octopus_slots already has - it adds no confirmation value there. That also explains why a counter-example is hard to spot by eye: under the default mode the sensor's state changes in lockstep with the schedule, not with reality, so a slot that later gets revoked still shows the sensor "on" right up until it's pulled.

The good news: the same entity also carries a started_dispatches attribute, populated unconditionally from Octopus's raw data regardless of which mode is selected - only the derived on/off state depends on the mode, not the attribute. Stage 1's diagnostic (the dispatch-timeline logging in this PR) already reads that attribute directly. So the fix isn't "trust the sensor's state" or "make everyone check their integration settings" - it's to key any future signal off started_dispatches's actual content, which is mode-independent by construction.

That still leaves the attribute's own real-world reliability (lag, a planned dispatch jumping straight to completed without ever showing up as started - both raised earlier in this thread) genuinely open. That's exactly what we're hoping the dispatch-timeline logging in #4528 will let us watch for over the next while, rather than guess at.