diff --git a/draft/keyupdate-draft.md b/draft/keyupdate-draft.md new file mode 100644 index 0000000000..ca577d7a5a --- /dev/null +++ b/draft/keyupdate-draft.md @@ -0,0 +1,361 @@ +# A keyupdate push channel to maintain reliable chat connectivity + +[Multi-relay support for chat profiles was added in March 2026][multi-relay], +allowing them to use multiple relays for receiving and sending messages. +While instant onboarding is being extended to multi-relay onboarding ([#8444]), +adding and removing relays automatically is not settled, +not least because changing a relay is unsafe today. +This draft proposes a *keyupdate push channel* +that shares our current key with our contacts when it changes, +without waiting for a chat interaction. +It helps keep chats connected now, +and makes automatic relay changes safe enough to design later. +The draft stays with the concept: +how such a message is encrypted and addressed is left to the two +implementations summarized in [Two ways to send a keyupdate](#two-ways-to-send-a-keyupdate). + + +## Problems of maintaining reliable chat connectivity today + +A profile's relay list lives inside its own key, as a signed notation that travels with the key. +A contact sends to the addresses contained in the key, signed by the key holder. +Rooted in the [Autocrypt 1](https://autocrypt.org) inline key-distribution specification, +there is no central directory, no probe and no removal notice, +so a contact's copy is only refreshed by traffic: +a message we send them, or a group message from someone else +that gossips our key onwards. + +Reachability therefore decays exactly where traffic is thin. +Three situations turn that decay into a cut conversation, +or into costs of preventing one: + +- **Mutual silence.** + If neither side has written since our relays changed, + the contact keeps the old list indefinitely. + Their next message goes to a relay we no longer read, + and they get silence rather than a bounce because the old account still exists. + Even if a bounce comes, it's hard to effectively act on it for users. + [#7878] is a user report of exactly this, and the reporter's objection + is the design point: + "people changing relays shouldn't need to know to write in every room + they're in just to make sure everyone gets the update". + It was closed expecting [#7865] to fix it, and [#7865] did land: + the relay list is now carried in the key. + That settled what gets distributed, not when, + which is the part still missing. + +- **Deliberate relay removal.** + Users remove relays for good reasons: + privacy, distrust of a provider, or wanting to use relays they chose. + Here the cut would be immediate and total: + every contact still on the old list would write to an address nobody reads. + It does not happen today only because core doesn't really remove a relay, + preventing lost messages at the costs of the "unpublished relay" workaround below. + +- **Adding relays during degradation.** + Relays sometimes get added precisely when the existing ones stop working, + so the announcement would have to travel + at the moment reachability is already impaired. + A contact who does not learn the new relay + keeps writing to an address that may already be dead. + + +What exists today only narrows the gap. +MDN key refresh ([#8481]) covers contacts who read our messages without +replying, so the exposed population is mutual silence +rather than all quiet contacts. +Unpublished relays buy time, and only for removal. + + +### The "unpublished relay" workaround is its own problem + +"Remove" today marks a relay *unpublished*: the address stops being advertised, +but the relay keeps a full IMAP connection and keeps receiving. +This is deliberate ([#8384]) and it does buy safety for the cases above. +The price is that users cannot end their relationship with a relay: + +- **Retention is unpredictable in both directions.** + Cleanup requires 90 days both since the removal itself + and since the last message received there, + and every fetched message resets the second clock, + with no filter for blocked contacts or spam, + so a removed relay that keeps getting mail is kept indefinitely. + Adding a relay at the five-relay cap does the opposite + and hard-deletes the least recently used unpublished one at once. + +- **A relay the user removed keeps being used.** + Unpublished transports get an IMAP loop like any other + and count against the same cap, + so a profile can hold connections to five relays, + four of them ones the user pressed "Remove" on. + Chatmail relays only delete accounts after a period without any login, + and core keeps logging in, + so the account there outlives the removal as well. + +- **Nothing the user can do shortens it.** + Real deletion is documented as internal and test-only. + Users are told at removal time that the relay is phased out, + but not for how long, + and not that we keep connecting and receiving there. + [#8384] records the friction from a user: a removed relay still advertising + TURN servers, and a permanent connection error with "no way to get rid of + it", patched as a symptom in [#8550] while the relay itself stayed. + +This grows with multi-relay setups, and multi-relay onboarding ([#8444]) brings them. +The [privacy notes being drafted for the apps][privacy notes] +would somehow need to describe this behaviour, and the description would read badly. + +Unpublished relays exist to protect exactly the contacts +the new keyupdate push channel would reach, +so the keyupdate push channel would let us drop the concept completely. + + +## Moving on from Autocrypt1: A keyupdate push channel + +We should be able to hand our current key to our contacts +without waiting for a conversation to happen. +A keyupdate is a message that does exactly that: +it carries our current key with its signed relay list +to the contacts who still hold an older copy, +sent when that list changes rather than when a chat happens. + +Whichever way such a message is sent, the same things follow from it: + +- **Contacts acquired at any point in the past are reached**, + and not only the ones we happen to write to, + given they have updated to a post-keyupdate app release. + +- **Delivery does not have to be complete to be useful.** + In group chats, cooperative Autocrypt gossip spreads what arrived: + members who received the fresher key pass it on to the others, + whenever one of them next writes to the group. + +- **"Remove" can mean actually removed.** + Pushing keyupdates removes the last reason to keep "unpublished relays" around, + so removing a relay can do what users intend it to do. + +- **Automatic relay management becomes easier to design.** + What ships under that name today is initial onboarding only + ([#8444], still off by default): no rotation, no removal. + Designing those proved to be hard while changing a relay is unsafe, + because any automatic change would silently cut off + the contacts who do not hear about it. + Keyupdates would lift that constraint and could land in the next release, + well before automatic addition/removal mechanics are settled. + + +## Constraints of keyupdate messages, metadata and processing + +- **A keyupdate carries the key and nothing else.** + The keyupdate push payload is "contacts-public": + it is the key and its relay list, + the same data we hand out in every chat where we participate. + But a keyupdate is not a conversation, + and it travels automatically to an audience nobody picks per message, + so **the keyupdate channel must never carry user generated data + or metadata besides the public key itself and it must not reveal + to contacts the identity of other contacts.** + +- **It is end-to-end encrypted.** + In chatmail clients "public" keys are by default hidden identities, + only transmitted in encrypted messages, with the goal of preventing + a curious or abusive relay operator to track identities. + +- **It goes to contacts, not to subscribers, and not to our own devices.** + The recipients are unblocked key-contacts we share an accepted 1:1, + group or subscribed-channel chat with. + Subscribers of our own channels should be left out, + because they could be a big number and it doesn't contribute to better chat connectivity. + Our own devices learn relay changes through regular multi-device sync, + because changing transports involves private credentials + for accessing a relay address, + and because we perform some merging on concurrent transport additions. + +- **It is accepted from any unblocked key-contact.** + The relay list lives in a direct key self-signature, + and certificate merging verifies it and prefers the newest one, + so merging, and not the way a keyupdate arrived, + is the cryptographic gate for every certificate we get. + A stale relay list is worth updating in any case, + whatever chat state we have with that contact. + +- **Replay changes nothing.** + An old update can be replayed forever, + but certificate merging keeps the direct key signature + with the newest creation time, on a tie the one already stored, + so a replay can not revert a relay list. + The rest of the certificate is fixed today: + a key carries one encryption subkey that never rotates. + Rotating Autocrypt 2 subkeys might need a fresh look at this. + +- **It is invisible on arrival.** + The key should be applied on the normal Autocrypt path + and the message then trashed rather than filed: + no chat, no counter, and no refresh of the sender's "last seen", + because an invisible message should not light up an online dot. + +- **It stays quiet on clients that do not know it yet.** + No new chat and no contact request should ever be created there. + Old and new clients can therefore ship together, + with coverage growing as clients update. + + +## When and how a keyupdate goes out + +Sending should be driven by a diff, not by an event. +A device records the relay list it last announced, +and a keyupdate is due only when the current relay list differs from it. +That single decision gives us the rest: + +- **Debounced.** + Changes within a short window should go out as one message, + so an automatic relay add/remove yields one keyupdate, + and an add followed by a removal of the same yields none. + +- **Short debouncing for user observability**: + Around 30 seconds may be a good enough debounce window to start with. + It is long enough that several changes usually travel in one keyupdate + (current initial onboarding adds up to three relays sequentially), + and short enough that someone changing relays by hand + can still watch the effect arrive with chat peers, + which is worth more than optimising against a few small extra messages. + +- **Nothing on upgrade.** + Existing profiles must start with their current relay list + already recorded as announced. + Otherwise the keyupdate release would send unnecessary noise + without any actual relay changes. + +- **Exactly one device in a multi-device setup announces.** + A device applying a relay change received over + multi-device sync should record the resulting list as announced without sending. + That also systematically prevents a device catching up on a backlog of old sync + messages from announcing historical states. + +- **Behind real traffic.** + The message should best leave from the SMTP loop once its queue is drained, + so a keyupdate never delays a user message + and is only attempted on a connection that just proved to work. + + + +## Two ways to send a keyupdate + +Nothing above says how a keyupdate reaches its recipients, +and two implementations currently explore that question: + +- [#8601] sends a single symmetrically encrypted message + to all recipients at once, re-using the broadcast machinery core has, + with the secret derived from our own key + so that everyone already holding a copy computes the same value. + Nothing in the message is recipient-specific + and its cost is largely independent of the recipient count. + Wire format, secret derivation, audience and sending policy + are documented in its `src/keyupdate.rs` module docs. + +- [#8588] (WIP) sends ordinary asymmetrically encrypted messages, + addressed per recipient in chunks of a few dozen contacts, + staying on paths core already has + at a cost that grows with the number of contacts. + +Both fit the concept described here, +and they trade off differently on who can decrypt a keyupdate, +what a relay observes, and how much machinery is involved, +which is a discussion for the PRs rather than for this draft. + + +## Open questions + +- Should a keyupdate be rate limited for a profile whose relay list flaps on its own? + Manual fiddling is bounded by the person + doing it, but future automatic relay add/remove needs to think about limits. + +- What should happen to delivery status notifications for dead addresses? + They arrive per address and refer to one Message-ID + but we don't do much with them. We could probably evolve to exclude such + bounced addresses from future key updates but it shouldn't block + a first key update implementation. + +- Is the sending set right? Narrowing the keyupdate recipient set + by activity would shrink what we send, but it would also disclose + to the relay which of our contacts are close ones. + Note that a quiet contact is indistinguishable from one who left, + so dropping them silently loses their next message. + + +## What a reliable keyupdate rollout might open up + +None of the following sections is proposed here. +But each of them becomes thinkable once changing relay setups is safer. + + +### Maximizing reliable deletion ("forward secrecy") with Autocrypt 2 + +The keyupdate channel is not necessarily only about relays. +Autocrypt 2 ([#8317]) introduces expiring, rotating subkeys, +whose distribution has a similar shape as the problem above: +a contact we never write to ends up not having our "reliable deletion" encryption keys. +The described keyupdate mechanism could help, +or at least it could play into the cadence of updates ... + + +### Forward-rotating through the relay network + +Today a relay change feels like something to be survived. +If keyupdates make it routine, it becomes something we could choose to do: +Better relays appear, and a profile could drift towards them +instead of staying wherever it happened to be born. +A safe frequency is capped by how reliably keyupdates land, +which is the interesting part: +rotation is exactly as feasible as the keyupdate channel is reliable. + + +### Immortal chat connectivity: surviving total relay loss + +One case is already out of reach of anything we send directly. +In [#8329] both sides lost all their relays at once, +so any announcement would go to addresses the chat partner can not access anymore. +Cooperative Autocrypt gossip can still bridge that +if some mutual contact remains reachable from both sides, +which is what the reporters ended up relying on, +and failing that an out-of-band re-exchange restores contact, +which works since [#8358], rerunning the full securejoin protocol +when the address is outdated. + +Harder still is a whole region losing all its relays at once: +nobody can announce anything to anybody, no push channel helps, +and there may be no mutual contact left to bridge it. +But new relays, for example in sprouting mesh networks, might become available +and wouldn't it be useful to re-establish chat connectivity with those +who might be able to help you, or where you can pool resources? + +Interestingly, a keyupdate does not have to be addressed to anyone in particular. +Sent as one ciphertext readable by whoever holds the sender's key ([#8601]), +it does not have to be directly delivered to be useful: +something parked now can be picked up later. +Making it discoverable without handing everyone a way to enumerate and track profiles +is an interesting enough challenge to make cryptographers have exciting discussions. +But during emergencies people really just want things to work +and we could ask users in some way "It seems you lost access to all chats. +Do you want to enable emergency recovery mode?" + +We could then ephemerally park keyupdates at all accessible relays, +and scan what relay-stored key updates are decryptable for us. +As soon as chat connectivity recovers for first bunches of people, +they spread it further through cooperative Autocrypt gossip, +re-establishing chats over time, scaling chat connectivity for everyone. + + +[#7865]: https://github.com/chatmail/core/issues/7865 "Distribute information about relays in the key signature" +[#7878]: https://github.com/chatmail/core/issues/7878 "DeltaChat seems to sometimes propagate the incorrect relays for users who switched relays" +[#8317]: https://github.com/chatmail/core/issues/8317 "Autocrypt 2 support" +[#8329]: https://github.com/chatmail/core/issues/8329 "Scanning QR for existing contact doesn't update relay list" +[#8358]: https://github.com/chatmail/core/pull/8358 "fix: Rerun the full securejoin protocol if the address was outdated" +[#8384]: https://github.com/chatmail/core/issues/8384 "Remove hidden relays automatically" +[#8444]: https://github.com/chatmail/core/pull/8444 "feat: Basic multi-relay onboarding" +[#8481]: https://github.com/chatmail/core/pull/8481 "Improve and speed up autocrypt/pgp gossipping with MDNs" +[#8550]: https://github.com/chatmail/core/pull/8550 "fix: multi relay connectivity" +[#8588]: https://github.com/chatmail/core/pull/8588 "feat: Key update messages" +[#8601]: https://github.com/chatmail/core/pull/8601 "feat: introduce keyupdate message to inform contacts about relay changes" +[multi-relay]: https://delta.chat/en/2026-03-31-zero#maximizing-availability-and-resilience-through-multi-path-delivery "Maximizing availability and resilience through multi-path delivery" +[privacy notes]: https://github.com/deltachat/deltachat-pages/pull/1385 "Privacy notes being drafted for the apps"