Backport Asynchronization Series from Upstream to Align Patchset

[phip1611]

This first reverts everything building on our fork's asynchronization commit series, then applies the incompatible (and better) upstream solution and then reapplys our commits with minimal changes to make them compatible with the new base.

This is an effort to prepare the v54 bump and to test that upstream code still passes our test suite. This backport should be by far the most invasive backport.

Closes cloud-hypervisor#697.

Patchset was created with AI assistance.

libvirt pipeline: https://gitlab.cyberus-technology.de/cyberus/cloud/libvirt/-/merge_requests/294

Hints for Reviewers

Details

Branch gardenlinux-backport-asynchronization, 83 commits on top of
68e6d4ed7 ("hypervisor: Set fw_cfg as default feature").

Structure of the series

The series has four blocks. Reading it in order is the intended way to review
it; the net diff against 68e6d4ed7 is confined to 13 files and is a better
way to judge the end state.

1. Commits 1-40: reverts, newest fork commit first. Everything that builds
   on the fork's migration worker is reverted before the worker itself, so each
   step stays compilable. The naive order (async core first) dead-ends at
   034935aed; ~27 dependent commits have to go first.
2. Commits 41-47: upstream cherry-picks - 15cab7ee6, e03456769,
   5d835bdff, 796fc055b, 7c7a827de, a56594324, 6a16b65ea.
3. Commits 48-82: the fork's features re-applied on upstream's worker, in
   their original chronological order, plus a few align/port commits.
4. Commit 83: a comment-only correction in vm-migration/src/protocol.rs.

Commit message conventions

• Cherry-picks carry, between subject and body:
  Cherry-picked from upstream commit <sha> ("<subject>").
  A following paragraph names the fork-specific adaptation, if any was needed.
• Re-applied fork commits are titled <component>: Reapply "<original subject>" and their body starts with This reverts commit <revert-sha>.,
  i.e. they point at their own revert in block 1. An "Adapted to upstream's
  migration worker: ..." paragraph explains every deviation.
  (git's own default is Reapply "<subject>", but scripts/gitlint/rules
  requires a component prefix, hence the prefixed form.)
• Four titles were shortened to fit 72 columns; those bodies name the original
  subject verbatim.
• Re-applied commits keep their original author and that author's
  On-behalf-of/Signed-off-by trailers, with a second Signed-off-by added.

Deliberately not re-applied

• a904e9b9d, 034935aed, e3977923b, f7303f34a, 3b3eda17b,
  2a2f4e618, c70dd7790 - the fork's async core; upstream's series
  implements the same thing.
• 27ba41b55 ("limit to running VMs only") - upstream's worker carries the
  initial VM state and resumes after a failure only if the VM was running, so
  the fork-only guard is redundant. Migrating a paused source VM works
  again; the revert therefore also adds the missing "don't pause twice"
  guard to do_memory_migration(), as upstream has it.
• 838ea2e71 ("streamline migration failure cleanup") - upstream's worker
  already has the resulting shape.
• cf1874233 ("switch to non-blocking SendMigration call") - upstream's
  worker already dispatches without blocking. Only the fork-only keep_alive
  option survives, as the separate commit "vmm: keep the VMM alive after a
  migration if requested".

Where the fork features hook into upstream's worker now

• VmOwnership::Migration gained device_manager: Weak<Mutex<DeviceManager>>
  next to upstream's vm_info_response, so the epoll thread keeps draining
  virtio activations. This fills upstream's // TODO: ... virtio activation handling while migrating.
• The postponed reboot/shutdown state is passed through
  MigrationWorker::spawn(); the Exit/Reset/GuestExit arms of the
  control loop fill upstream's two lifecycle // TODOs.
• Cancellation: the Arc<AtomicBool> lives on MigrationWorkerHandle and is
  reached through VmOwnership::Migration, so the fork's separate
  Vmm::migration_thread_handle field is gone (it only existed because the
  flag could not be reached through the old enum). vm_cancel_migration() is
  panic-free now.
• MigrationWorkerResult carries the migration config back to the VMM thread,
  which is how check_migration() evaluates keep_alive.
• The progress snapshot stays the global MIGRATION_PROGRESS_SNAPSHOT. It is
  installed after the last fallible step before the hand-over and is marked
  failed when spawning the worker fails, so it can no longer stay Ongoing
  forever.

Behaviour changes worth a close look

Everything below is either upstream's behaviour or follows from the decisions
above - none of it is accidental, but all of it is externally visible:

• Paused source VMs can be migrated again (see 27ba41b55 above).
• After a successful migration the source VM is shut down explicitly
  (vm.shutdown()), which is upstream's code. It resumes device workers before
  joining them, at a point where the disk locks are already released and the
  destination has taken over, and the control loop now blocks on those joins.
  Upside: with keep_alive=on the fork used to leak the detached threads.
• vm.deleted event: now emitted after the shutdown instead of before, and
  also for a created-but-never-booted VM (upstream's vm_delete).
• A cancelled migration now replays a postponed reboot/shutdown. The replay
  sits inside try_resume_vm_after_failed_migration(), which is what the fork
  did too (its try_resume_vm contained the replay); the .context() strings
  still say "after failed migration" for that reason.
• vm_info during a migration returns the snapshot captured at dispatch
  time, so device_tree is populated and state is the pre-migration state,
  instead of the fork's hardcoded Running + device_tree: None.
• vm_boot/vm_restore error variants follow upstream (VmAlreadyCreated,
  VmMissingConfig), and vm_add_generic_vhost_user is no longer the fork's
  unimplemented!() stub (its #[ignore]d unit test runs again).
• Three seccomp rules (rt_sigprocmask, getcwd, clock_nanosleep) leave
  http_api_thread_rules(); they were added only for the blocking
  send-migration call. SYS_read stays.
• ch-remote send-migration is unchanged: it still waits by default and
  polls vm.migration-progress, with --dispatch to opt out. Upstream's test
  commit 6a16b65ea assumes fire-and-forget, so the TCP timeout test passes
  --dispatch (separate commit).

Unchanged on purpose (safe to skip while reviewing)

• vmm/src/migration_transport.rs -> vmm/src/migration/transport.rs is a
  pure rename, zero content change: keep-alive stream, TCP-thread
  cancellation, worker-failure fixes, thread cleanup are all untouched.
• cloud-hypervisor/src/bin/ch-remote.rs, vmm/src/vcpu_throttling.rs, the
  TLS stack and all of vm-migration/ except the protocol comment are
  byte-identical to 68e6d4ed7.
• No event was renamed, added or dropped (only vm.deleted moved, see above),
  and the wire protocol is unchanged - Command::KeepAlive = 8 /
  CompletePaused = 9 stay, so migration between fork builds keeps working.

Align/doc commits added on top of the pure revert/pick/reapply work

• openapi, docs: document the migration-progress endpoint
• vmm, openapi: document the keep_alive send-migration option
• openapi, docs: document the cancel-migration endpoint
  (these three sit directly after the commit that introduces the respective
  surface; the endpoints were previously in neither docs/api.md nor the
  OpenAPI spec)
• vm-migration: describe the discriminant collision completely - comment
  only; records that upstream also uses discriminant 9 (PageFault), which
  this branch does not need, and that fork <-> upstream migration is therefore
  knowingly impossible.

Verification performed

• cargo check --package cloud-hypervisor on every commit of the series
  (the branch is bisectable; cargo check -p vmm alone is broken in this repo
  because vmm's default features are empty).
• On the tip: the CI jobs Clippy (kvm), Clippy (mshv), Clippy (mshv+kvm),
  Clippy (tdx,kvm), Clippy (default features), Build (mshv),
  cargo clippy -p cloud-hypervisor --test integration (that file is
  cfg-gated, a plain check compiles it to nothing), cargo test -p vmm --features kvm --lib (88/88), cargo +nightly fuzz check with
  RUSTFLAGS=-D warnings, and the OpenAPI validator CI uses
  (openapitools/openapi-generator-cli validate).
• gitlint with this repo's custom rules over all 83 commits.
• Integration tests and the external libvirt-tests suite were not run.
  The two changes most likely to show up there are the explicit source-VM
  shutdown and the vm.deleted event order.

Known open points

• VmSendMigrationData::keep_alive is the only field of that struct without
  #[serde(default)], so a JSON body omitting it is rejected, although the
  CLI syntax and the new OpenAPI entry treat it as optional. Pre-existing, not
  changed here.
• The vm.migration-progress and vm.cancel-migration endpoints still have no
  D-Bus counterpart; only the HTTP surface is documented.
• fuzz/Cargo.lock is stale at the base commit already (fw_cfg fallout). CI's
  fuzz job does not use --locked, so it does not fail.

---

Too be honest, seeing this, I am unsure if our approach is the right choice. My intuition tells me to recreate v54 directly commit by commit (or commit series by commit series) rather than investing time into the gardenlinux branch. But I am happy to stick to our plan. Just unsure if it still brings us anything. I had to revert the world to make this possible.

However, once we merged this, the other backports are far simpler.