Skip to content

fix(ui): stop rebuilding the socket on every token refresh - #9540

Merged
lstein merged 1 commit into
invoke-ai:mainfrom
lstein:fix/socket-rebuild-on-token-refresh
Aug 25, 2026
Merged

fix(ui): stop rebuilding the socket on every token refresh#9540
lstein merged 1 commit into
invoke-ai:mainfrom
lstein:fix/socket-rebuild-on-token-refresh

Conversation

@lstein

@lstein lstein commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

The socket.io connection was keyed on the bytes of the auth token. The sliding-window
middleware mints a replacement token on every mutating request, and the client commits one
roughly once a minute (TOKEN_REFRESH_THROTTLE_MS = 60_000), so during any activity
useSocketIO's connect effect re-ran on a fixed 60s cadence:

  • disposeEventListeners() tore down every app-wide socket listener,
  • $lastProgressEvent / clearAllProgressEvents() wiped in-flight progress,
  • the socket was disconnected and a new one built.

Visible symptom: a preview glitch mid-generation once a minute, and a matching
Socket … disconnected (… reason: client disconnect) / connected pair in the server log:

[16:26:36] Socket HFyPd7AzyG8Og1s-AABN disconnected (…, reason: client disconnect)
[16:26:36] Socket 1L38E1XFMEf6cy6RAABP connected with user_id: …

Nothing about a refreshed token changes who is connected or which rooms they belong to, so the
rebuild bought nothing.

Change

  • authSlice: getTokenSessionKey() / selectAuthSessionKey — the session a token belongs to:
    its user_id claim plus the token_epoch revocation claim, falling back to the token's own
    bytes when it carries no user id.
  • useSocketIO: the options memo — and therefore the connect effect — keys on that session key
    instead of the token. A logout, a session expiry, another account taking over the tab, or a
    password change that supersedes the epoch all still change the key and still rebuild the socket.
  • auth is now passed as a callback. socket.io invokes it in Socket.onopen on every
    connection attempt (socket.io-client/build/esm/socket.js:185-194, 405-412), so a socket that
    now outlives the token it was built with still presents the live one on any reconnect.
  • The Authorization extra header is dropped. It was the server's fallback for a handshake
    carrying no auth payload (sockets.py:239-246), but a header is fixed for the life of the
    manager, so it could only ever hold the token that was live at socket-build time — i.e. offer a
    credential the client may already have discarded. The payload is read live and cannot go stale.
  • An io server disconnect now drives a bounded, backed-off reconnect of the same socket
    (5 attempts, 1s doubling; cancelled on teardown).

Why the epoch, and why the reconnect

Both come from the same server behaviour. _handle_user_access_changed force-disconnects sockets
when a user is deactivated or when their token epoch is superseded (sockets.py:761-767), and a
server-initiated disconnect is terminal for socket.io — Socket.ondisconnectdestroy()
Manager._close() sets skipReconnect (socket.io-client socket.js:613-616,
manager.js:280-283). Under the old byte-keyed effect the client happened to recover because the
replacement token changed the memo; keying on identity alone would have removed that.

  • The epoch covers the password-change case, which the server documents as
    "the session that performed the change reconnects with its replacement token"
    (sockets.py:691-696). Replacement and sliding-window tokens both carry token_epoch straight
    from the user record (auth.py:52-58, api_app.py:177-183), so a routine refresh keeps the key
    stable while a revocation changes it.
  • The bounded reconnect covers the deactivation branch, which changes nothing the client keys on.
    A deactivation that is reverted moments later would otherwise leave that tab with no events and
    the Invoke button disabled (via $isConnectedreadiness.ts) until a page reload. If the
    client is genuinely no longer welcome, the reconnect is rejected with a connect error, which
    socket.io does not retry — so the retry cannot become a loop.

Room re-subscription is unaffected: subscribe_queue / subscribe_bulk_download are emitted
from the connect handler (setEventListeners.tsx:106-111), which fires again on every
reconnect.

Tests

useSocketIO.test.tsx — mounted-DOM coverage (happy-dom + act, the pattern from #9439):

  • a sliding-window refresh keeps the live socket, and a later reconnect presents the refreshed
    token;
  • a replacement token with a bumped epoch rebuilds it;
  • a same-tab setCredentials to a different user rebuilds it;
  • an externalTokenAdopted from another account tears it down and waits for /me to rehydrate;
  • logout tears it down;
  • io server disconnect reconnects the same socket, transport close is left to socket.io, the
    retries are bounded, and a pending retry never revives a socket the session has discarded.

Plus session-key units in refreshedTokenConsumers.test.ts.

Every behaviour was mutation-checked one at a time — each mutant fails exactly its own test and
nothing else.

Verification

Ran the fixed build against a live server for ~4 minutes with pnpm dev, sampling the server's
TCP connections at 0.5s: the connection carrying the fixed client's socket stayed established in
285/285 samples, across three of the minute boundaries that previously dropped it.

Full frontend suite green (166 files / 2199 tests); all five pnpm lint:* clean.

Two independent fresh-context adversarial reviews were run against the change (state lifecycle /
existing-caller assumptions, and auth + multi-user isolation); the epoch keying, the header
removal and the bounded reconnect are all responses to their findings.

The socket.io connection was keyed on the auth token's bytes. The sliding-window
middleware mints a replacement token on every mutating request and the client
commits one once a minute (TOKEN_REFRESH_THROTTLE_MS), so during any activity the
connect effect re-ran on a 60s cadence: listeners disposed, progress store
cleared, the socket disconnected and rebuilt — a visible preview glitch
mid-generation, and a "client disconnect"/"connected" pair in the server log
every minute.

Key the socket on the session the token belongs to — its user id and revocation
epoch — instead. A routine refresh keeps the key stable; a logout, an expiry,
another account taking over the tab, or a password change that supersedes the
epoch all still change it and still rebuild the socket.

The epoch belongs in the key because the server force-disconnects sockets whose
epoch is superseded and expects the client to come back with its replacement
token, and a server-initiated disconnect is terminal for socket.io: the client
sets skipReconnect and never retries. For the same reason, an `io server
disconnect` now drives a bounded, backed-off reconnect of the same socket — the
deactivation branch of _handle_user_access_changed changes nothing the client
keys on, so without it a reverted deactivation would leave the tab with no
events and Invoke disabled until a page reload.

`auth` is passed as a callback so each connection attempt presents the token
that is live then, rather than the one the socket was built with. The
Authorization extra header is dropped with it: a header is fixed for the life of
the manager, so as the server's fallback it could only ever offer a credential
the client might already have discarded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YbMt7bvKerUbuR5r1zQQtt

@JPPhoto JPPhoto left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No merge blockers. You can decide if this is worth addressing or not:

  • invokeai/frontend/web/src/services/events/useSocketIO.ts:129: socket.connect() leaves Socket.IO Manager reconnection enabled; a transport failure during custom retry bypasses five-attempt bound and retries indefinitely. Effect: persistent retry traffic while unavailable. Likelihood: plausible server/network failure after server disconnect. Recovery: reload/teardown stops it. Test: real socket.io-client 4.8.3 produced 4 reconnect attempts in 180 ms against a closed port; bound Manager retries explicitly.

Suggestions:

  • Instead of relying on Manager defaults, coordinate or disable Manager reconnection during custom retries so the five-attempt limit is enforceable.

@lstein
lstein merged commit fcb437e into invoke-ai:main Aug 25, 2026
17 checks passed
@lstein
lstein deleted the fix/socket-rebuild-on-token-refresh branch August 25, 2026 22:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.14.1 frontend PRs that change frontend files

Projects

Status: 6.14.1: Bug fixes to 6.14.0

Development

Successfully merging this pull request may close these issues.

2 participants