Design: render priority as a first-class concept (dueAt + priority), replacing five overloaded scheduling mechanisms

[harper-joseph]

Problem

The render queue has no concept of priority. claim is strictly nextRenderTime-ascending, so under
any capacity deficit the queue serves whatever is oldest-due — regardless of whether a page was
submitted by the site owner or discovered by a third-party crawler, and regardless of how tight its
freshness budget is.

Measured on a production deployment during a multi-hour backlog: ~46% of the overdue queue was
bot-discovered rather than sitemap-submitted. Roughly half the render capacity was being spent on
pages the site owner never submitted, while submitted pages aged past their SWR window and fell
through to origin.

There is no way to express "this matters more" today. Instead, five separate mechanisms overload the
same conflated field to nudge scheduling:

mechanism	how it expresses intent
documented nextRenderTime = 1 trick	writes a sentinel due time to jump the queue
Target.revalidate	writes Date.now() into expiresAt
bulk invalidation epochs	deliberately does not touch the queue (cadence-heal only)
render.failureRetry (incl. nonSitemapPenalty)	pushes due time forward as a penalty
demand ladder	reallocates cadence within the route's budget

Each is individually reasonable. Together they mean scheduling intent is scattered across five homes,
none of them named "priority", and every new requirement overloads nextRenderTime again. A sixth
encoding would make this worse, so the API should be settled before an implementation is chosen.

Proposal

1. dueAt and priority are separate concepts

They are conflated today, which is the root of the problem above.

concept	means	derived from
dueAt	when this content should be refreshed — a freshness deadline	interval / TTL / cadence
priority	how much it matters to hit that deadline when capacity is short	provenance, operator intent, health

Truth about content vs. truth about importance. Neither is ever encoded into the other at the API
level.

2. Queue order is (priority class, then RELATIVE lateness)

Relative lateness = (now - dueAt) / interval, descending.

Relative rather than absolute is what makes "shorter TTL should win" fall out for free: a 6 h page
3 h late is 50% late, a 48 h page 3 h late is 6% late. No TTL-specific rule, no extra class, no
bucketing scheme, and it stays correct when route cadences change.

3. priority is an ordered enum, and it is DERIVED, not stored

urgent      operator intent — manual clear / invalidate / force-render
submitted   present in a sitemap
discovered  crawler-found, never submitted
cold        repeatedly failing, or never successfully rendered

Named and ordered rather than a free-form integer: self-documenting in logs and the admin UI,
reviewable in a diff, and it cannot drift into arbitrary magic numbers.

Derived is load-bearing. Compute the class at schedule time from stable inputs (presence of
sitemapUrl, route match, failure count) instead of storing it. A config change is then retroactive
on each key's next render with no sweep of the corpus — the same discipline renderInterval
precedence already uses (route > stored > default, resolved at schedule time). Store it only as a
derivation cache if profiling demands it, exactly as demandInterval does.

4. Fairness is scheduler policy, NOT part of the ordering key

render:
  priority:
    fairness:
      minShare: { discovered: 0.10, cold: 0.02 }

A strict class ordering starves lower classes for the whole duration of a deficit. Their cached pages
then pass expiresAt + swrTtl and their traffic shifts to origin — which is correct but converts a
render-capacity problem into an origin-load problem.

Keeping the fairness bound out of the ordering key means it is tunable live, with no rewriting of
stored rows. Any scheme that bakes fairness into the key (time-bucketing, quantised composite keys)
requires re-encoding every stored value to change the bound. This is the strongest single argument for
keeping ordering and fairness separate.

5. One operation replaces the ad-hoc tricks

POST /render_queue/prioritize   { url | scope, class: urgent }

This turns the cache-clear case into explicit policy rather than an accident of which mechanism the
operator reached for:

• clear-and-rush — drop the cached page and re-render at urgent
• clear-and-wait — drop the cached page, let normal cadence refill it, serve origin meanwhile

Both are legitimate; today the choice is implicit. It also retires nextRenderTime = 1, which is a
documented hack that only works if you happen to target the owner node.

6. Observability is part of the API, not an afterthought

• backlog depth per priority class
• priority as a dimension on the render metrics

Without this there is no way to answer "is prioritisation working". Note the existing constraint that
recordAnalytics allows exactly three dimensions per metric — so this likely needs its own metric
name rather than an extra dimension on an existing one.

Invariants any implementation must hold

1. expiresAt always reflects true dueAt. Never a priority-adjusted value. Violating this
   serves pages as fresh when they are not — silent and severe.
2. The claim floor is per-class (or otherwise partitioned). A single watermark shared across
   classes lets one low-priority row pin the entire queue.
3. Nothing outside the queue module reads the raw sort key. Callers see dueAt and priority.
4. No corpus-wide sweep is required to change priority policy.

Implementation options — deliberately deferred

All of these satisfy the API above and are private to the queue module. Listed so the trade space is
recorded, not to pick one here:

option	priority strength	cost
additive offset on due time	bounded by the offset; absorbed once the backlog exceeds it	trivial; too weak
time-bucketing (bucket-major, class-minor within the bucket)	bounded by bucket size	spends due-time precision; unusable for routes whose interval approaches the bucket; couples bucket size to swrTtl
high-order class prefix on the sort key	unbounded	leaves the valid timestamp range for low classes, so floor/backlog/decode paths need class awareness
separate table or index per class	unbounded	extra index; class change becomes delete + put

Because the API hides the choice, the cheapest adequate encoding can ship first and be replaced later
without touching the serve path, admin UI, invalidation, or reconcile.

Open questions

1. Should urgent bypass the fairness reservation entirely, or compete within a reserved share?
2. Should cold be a priority class at all, or is that the demand ladder's job? (The ladder currently
   treats the route's base interval as its own top rung, so it can only make pages faster, never
   slower — a never-visited page sits at base forever. Extending it downward overlaps with cold.)
3. Does bulk invalidation default to clear-and-rush or clear-and-wait?
4. Is relative lateness the right within-class comparator for urgent, where absolute recency of the
   operator's request is arguably what matters?