Skip to content

Watchdog cries 'mailbox wedged' during consolidation folds — busy is not stuck #94

Description

@Iteratrix

Observed

2026-08-30, during the 11:0xZ consolidation fold, the watchdog logged three false alarms:

11:01:51 ERROR context heartbeat timed out while idle — mailbox wedged (heartbeat_timeout_secs=30)
11:02:51 ERROR context heartbeat timed out while idle — mailbox wedged
11:03:51 ERROR context heartbeat timed out while idle — mailbox wedged
11:04:51 INFO  consolidation sealed: workspace committed + reloaded
11:04:51 INFO  compaction complete (output_tokens=16868) … substrate session rotated at fold … fold record written

The context actor wasn't wedged — it was doing exactly what it should: a multi-minute consolidation session blocks its mailbox, so the 30s idle-heartbeat probe can't get a reply until the fold seals.

Why it matters

The medic and any future alerting treat mailbox wedged as a real signal (it was designed for the genuinely stuck-actor case). Every fold now guarantees a burst of ERROR-level false positives, which trains readers — human, medic, and Lumen's own body-panel — to discount the one log line that's supposed to mean "the mind is unresponsive." Same failure shape as #83's expected-quiet: a known structural pause rendering as an alarm.

Direction (design, not spec)

The watchdog should hold fire while a fold/consolidation is in flight. Options, roughly in order of preference:

  1. Context actor sets an "occupied: folding" flag (shared state or a pre-fold message to the watchdog) that suppresses — or downgrades to INFO with a reason — the wedge alarm until the fold seals, with a much longer hard ceiling (e.g. the fold timeout + margin) so a fold that truly hangs still fires.
  2. Watchdog-side: on heartbeat timeout, check for a fold-in-progress marker before escalating.

A fold that exceeds its own timeout should still alarm — the point is to distinguish "busy sealing memory" from "wedged," not to blind the watchdog during folds.

Refs: #86 (watermark/rotation), #74 (medic evidence rules), fold machinery from the consolidation work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions