perf: read workload statuses one prefix per app, entrypoint and node - #751
Open
CMGS wants to merge 1 commit into
Open
perf: read workload statuses one prefix per app, entrypoint and node#751CMGS wants to merge 1 commit into
CMGS wants to merge 1 commit into
Conversation
bindWorkloadsAdditions fetched every workload's status with its own GetOne; a listing of N workloads cost N status round trips on top of the meta and node reads. Workloads sharing an appname, entrypoint and node share a status key prefix, so one GetPrefix per group replaces them: N*M reads become M for M nodes. Groups read concurrently under an errgroup capped at 32. A status read failure now fails the call instead of leaving the status nil silently; a missing or unmarshalable status still leaves StatusMeta nil.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bindWorkloadsAdditionsfetched every workload's status with its ownGetOne, so a listing of N workloads paid N status round trips on top of the meta and node reads. Workloads that share an appname, entrypoint and node share a status key prefix, so oneGetPrefixper group replaces them: N*M reads become M for M nodes. Groups read concurrently under an errgroup capped at 32.Lane A/B (test cluster, release v0.1.5 vs this branch,
eru-cli workload list --pod testanderu-cli workload get <all ids>, 3 rounds interleaved with the order rotated, medians of 24 timed calls per arm):One node is a wash because the per-workload reads already ran in parallel under the pool; the gain is the etcd server work proportional to N.
Behaviour that changes:
GetWorkloads/ListWorkloadsinstead of leavingStatusMetanil silently; a missing or unmarshalable status still leaves it nil.GetWorkloadreads the status blobs of its whole app/entrypoint/node group (one round trip as before, more bytes).Gates: build, vet, full tests, lint, fmt-check, asl on linux and darwin green; comments +1 (the prefix's trailing slash).