sync to refactored break - #2
Open
brownzebra wants to merge 2476 commits into
Open
brownzebra wants to merge 2476 commits into
brownzebra wants to merge 2476 commits into
Conversation
…urce docs: document inter-release AIM catalog management
Cluster-auth has been removed from the stack. Replaced with envoy SecurityPolicy in the blueprint.
sanity check to get script if missing from release
fix(blueprint): vllm-sr remove cluster-auth references
Match hardwareFamilies empty vs profiles behaviour to legacy.yaml and profiles.yaml on main; drop the claim that empty list is unsupported. Co-authored-by: Cursor <cursoragent@cursor.com>
…yaml Empty hardwareFamilies is an unfiltered catalog (Instinct 0.11.1+ plus mixed bases), not the old generic release set. Update chart comments and operator docs to match. Co-authored-by: Cursor <cursoragent@cursor.com>
…urce docs: align AIM catalog lifecycle with in-tree chart templates
The body-aware extAuth on ai-gateway buffers the request body at maxRequestBytes=64KiB before authorizing header-less (model-in-body) requests. Envoy returns 413 for any larger body (skipping auth, and overriding failOpen), so standard OpenAI/Anthropic clients sending 200KB-2MB contexts are rejected gateway-wide, for every model. Parameterize as aiGateway.bodyAuthMaxRequestBytes (default 32MiB) so large-context and image-bearing inference turns are not rejected; the value sits under the 50Mi connection bufferLimit and is tunable per env. EAI-8489
Follow-up on the 32MiB commit on this branch. - 4MiB, not 32MiB. The ai-gateway-discovery authz pod that receives the buffered body is capped at limits.memory 256Mi, and an in-flight header-less request is held both by Envoy and by that handler, so 32MiB is more than the current deployment can absorb. 4MiB covers a ~200k-token turn (~1MB) with headroom. - Drive both charts from global.aiGateway.bodyAuthMaxRequestBytes. The gateway-scoped policy here is not the only cap on the header-less path: ai-gateway-discovery stamps per-model catch-all SecurityPolicies (sectionName route-not-found) that are rule-scoped and override this one per route. They carry their own ceiling, so the two must move together. - Render through int64 and fail on a non-positive value. A bare number in a values file is a float64, so the root wiring emitted 4.194304e+06 and int64 casts that to 0, which would have published maxRequestBytes: 0 instead of failing. The matching ai-gateway-discovery change lands in silogen/core.
Raise ai-gateway body-authz request-body cap to 32MiB (EAI-8489)
The AIM catalog docs described behaviour that only exists in unmerged work, so operators reading them would configure the wrong thing. - Drop AIM_HARDWARE_FAMILY auto-detection claims from the lifecycle doc and chart README; auto-detection is not in cluster-bloom main, where the setting has no default and is injected only when set. Document that an install leaving it unset lands on the unfiltered catalog. - Replace the "published into the chart" source-of-truth framing with the actual manual copy at release time, and drop the dead public aim-build link. Point operators at cluster-managed additions when they need an AIM version before the next release. - Note that aim-cluster-model-source-additional is not shipped and needs one-time operator setup. - Fix stale "legacy" naming in root/values.yaml after the legacy.yaml -> unfiltered.yaml rename. Co-authored-by: Cursor <cursoragent@cursor.com>
AIM Engine discovery is append-only, so dropping an image from a source filter leaves discovered models in place, while deleting the source CR garbage-collects them and can break running deployments. Empty filters are invalid (MinItems=1). Packaged source names are a stable API. Co-authored-by: Cursor <cursoragent@cursor.com>
…ities Operators see the packaged baseline as the Cluster Forge release they installed. Who authored the internal lists is not needed to operate a cluster. Co-authored-by: Cursor <cursoragent@cursor.com>
…talog_state docs(aim-catalog): describe shipped catalog behaviour, not pending work
The root Application is rendered two times. bloom renders it one time
with helm --set, and ArgoCD then renders the same chart again to manage
the app-of-apps. The second render reads only the value files, so each
value that was given with --set and is not repeated as a parameter is
lost, and the root app overwrites its own spec without it.
global.domain and global.clusterSize were lost this way. An empty
domain made envoy-gateway-config write the CoreDNS rewrite
`regex .*\.`, which matches every name, so each name in the cluster
resolved to the gateway service. The ArgoCD controller could not reach
its repo-server, all 39 applications stayed in ComparisonError, and no
secret was made. An empty clusterSize dropped values_<size>.yaml from
the value files, so the cluster got the wrong set of applications.
Only the small path is affected. With externalValues.enabled the two
values come from cluster-values/values.yaml in the in-cluster Gitea.
To see the difference:
helm template cluster-forge root --show-only templates/cluster-forge.yaml \
--values root/values.yaml --values root/values_small.yaml \
--set global.domain=1.2.3.4.nip.io | grep -A1 global.domain
The certgen hooks carry "helm.sh/hook" but no delete policy. ArgoCD reads a missing policy as BeforeHookCreation, so it deletes each hook before it makes the hook again, and it does that each time it resumes the operation. The certgen ServiceAccount is a hook with weight -1, so it must exist before the certgen job can make a pod. ArgoCD deletes that ServiceAccount on each resume. The Job object stays, but each attempt to make its pod fails, so the job never completes. envoy-gateway stays at PreSync with "waiting for completion of hook", the Gateway API CRDs are never installed, and each application that uses a Gateway, an HTTPRoute or a TLSRoute cannot sync. Measured on a one-node cluster with CLUSTER_SIZE small, and with the root app domain fix in place: the job stayed at 0/1 completions for 4 minutes 35 seconds, the namespace held only the default service account, no Gateway API CRD was installed, and the job controller repeated ten times: Error creating: pods "envoy-gateway-gateway-helm-certgen-" is forbidden: error looking up service account envoy-gateway-system/envoy-gateway-gateway-helm-certgen: serviceaccount "envoy-gateway-gateway-helm-certgen" not found Each cluster size uses this chart, but the failure is a race between the interval of two resumes and the time the certgen job needs. A one-node cluster with CLUSTER_SIZE medium completed the job before the next resume, with this change and also without it, so only the small path shows the failure. The change stays correct for each size. hook-succeeded keeps the hooks until the job completes, and removes them after it.
bloom does not bootstrap gitea when CLUSTER_SIZE is small. The task
clusterforge_setup.yaml holds `when: (CLUSTER_SIZE | default('medium'))
!= 'small'`, and bootstrap_gitea.yaml is the only thing that makes the
secret gitea-admin-credentials.
values_small.yaml enabled the gitea and gitea-config applications all the
same, so the pod waited for a secret that nothing makes. Measured on a
one-node cluster: `secret "gitea-admin-credentials" not found`, 255
restarts in 59 minutes, and the application stayed Degraded.
The small path reads the repository from GitHub, because
externalValues.enabled is false and clusterForge.repoUrl names GitHub.
It therefore has no use for the in-cluster Gitea, and nothing else in
values_small.yaml names it.
EAI-8229 Fix CLUSTER_SIZE small: keep global.domain, hooks, drop gitea
Move the how-to to aim_model_management.md and the policy guide to aim_catalog_lifecycle.md, and align H1 titles and README links. Co-authored-by: Cursor <cursoragent@cursor.com>
Adds the envoyproxy/ratelimit binary + Redis backing AI Gateway QuotaPolicy enforcement. Opt-in only: not referenced in any enabledApps list, add it to your own overlay to install. Configurable via values: - redis.enabled/url: bring your own Redis instead of the bundled one - redis.auth: existingSecret-based password, for bundled or external Redis - redis.persistence: opt-in PVC (default stays ephemeral, matches the controller's fail-open quota behavior) - ratelimit.replicas, image/tag, logLevel, resources for both components Service name/namespace are hardcoded, not templated: the AI Gateway controller's --quotaRateLimitServiceAddr default and xDS NodeID are fixed strings this chart must match
feat: envoy-ai-gateway-ratelimit component
Co-authored-by: Cursor <cursoragent@cursor.com>
Expand the pre-upgrade checklist with helm/kubectl checks, including an AIMService in-use grep, and document the shared aim-base-models source name. Co-authored-by: Cursor <cursoragent@cursor.com>
docs: rename AIM catalog guides to match their scope
Use values.yaml app versions as reference
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.