Back-port the key-only attributes search and optional criteria to the older clients (#40, #41) - #47
Merged
craigmcchesney merged 3 commits intoSep 10, 2026
Conversation
…n helpers (#40) The protos document an empty `values` list on every AttributesCriterion as a key-only / existence search: match every record possessing the key, whatever its value. dp-service implements exactly that -- four of the five criteria route to MongoQueryFilterBuilder.attributeFilter(), which returns Filters.exists("attributes.<key>") on an empty list rather than an $in: [] that would match nothing, and MongoQueryFilterBuilderTest covers that path. Five helpers made it unreachable, each raising on empty values: PvMetadataQuery.attributes, ConfigurationQuery.attributes, ConfigurationActivationQuery.attributes, PvQuery.attr, ConfigQuery.attr. They now take `values: list[str] | None = None`, matching the shape #6 already shipped on DataSetQuery/AnnotationQuery, so all seven attribute helpers spell the concept the same way. The empty-key rejection stays, and matters more on the two v2 selectors: the server does not validate the key there, so a blank one would reach Mongo as an existence test on "attributes." and silently match nothing. Why this was worth doing rather than left as a documented workaround: - conventions.md justified the guard as refusing "a criterion that would silently match everything". True of tags([]) and pv_name(), but false here -- a key-only attributes criterion narrows the result set. The rule was generalized onto the one criterion it does not describe. - pv-metadata.md carried a whole section teaching users to drop through to the raw protobuf classes, a snippet needing `# cookbook:no-mypy` because the escape hatch is not statically checkable. That section is now a positive recipe and the checker type-checks all 104 snippets, none skipped for this reason. The guard was not an upstream-driven decision: it arrived in c101964 (2026-07-14) applying one rule across pv_name/aliases/tags/attributes at once, and all five proto comments predate it. Five existing tests asserted the rejection and are inverted rather than deleted, each covering both spellings (`attributes("k")` and `attributes("k", [])`). 721 unit tests pass. Plan: plan/tickets/40/plan.md. Verified against dp-grpc 6dfff3f and dp-service fddf692. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
…hods (#41) Six methods took `criteria` as a required positional argument, a shape dating from when the server rejected an empty criteria list. The server now treats an empty list as match-all, so "browse everything, paged" is a legitimate call -- but from Python it read `iter_configurations([])`, which looks like a mistake at the call site and was documented nowhere. `criteria` is now optional on query_pv_metadata / iter_pv_metadata, query_configurations / iter_configurations, and query_configuration_activations / iter_configuration_activations, plus their three private request builders, matching the shape #6 shipped on datasets and annotations. `iter_pv_metadata()` is the browse-all form. Non-breaking: criteria remains the first positional parameter. Server behavior verified rather than taken from the ticket. Each validation site carries a deliberate comment, not merely a missing check (QueryPvMetadataJob.java:38 and the two siblings): // An empty criteria list is match-all by contract (#245), not an // error, so there is deliberately no list-level emptiness check here. Per-criterion validation is retained everywhere, so this loosens the list and not its contents. Paging still applies: DEFAULT_QUERY_LIMIT = 100 in MongoSyncAnnotationClient, applied unconditionally -- dropping the last criterion does not change the page size -- which is why the docstrings point at iter_* rather than a bare query_* for browsing. Two corrections to the ticket body, recorded in the plan: - The default page size is NOT configurable. It is a hardcoded private static final int with no config key; the proto's "server-configured default page size" wording is loose and is deliberately not repeated in the client docs. - dp-grpc #245 / PR #147 covers only the three metadata queries. Datasets and annotations got match-all from the earlier #132 (7b2ea35) plus dp-service #248. The net behavior is as the ticket says; only the attribution was wrong. The v2 query methods are deliberately excluded: QueryParams still requires a PV selector or config criteria, since a time-series query with no selection is unbounded rather than a browse-all. Also adds a _ActivationCriterion TypeAlias -- the fully-qualified name is 109 characters and overflows the line limit in an annotated parameter. Using TypeAlias rather than a bare assignment keeps mypy happy in an annotation position, and net drops one pre-existing mypy error. 725 unit tests pass; ruff clean; 107 cookbook snippets checked. Plan: plan/tickets/41/plan.md. Verified against dp-grpc 6dfff3f and dp-service fddf692. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
There was a problem hiding this comment.
🟡 Changes recommended
Two documentation nits remain: stale helper docstrings and invalid blank-key examples.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Back-ports key-only attribute searches and optional criteria to older client APIs.
Changes:
- Added key-only attribute searches.
- Made query criteria optional for match-all browsing.
- Added tests and updated documentation, plans, and conventions.
File summaries
| File | Reviewed changes |
|---|---|
tests/unit/test_query_client.py |
Added coverage for key-only attribute criteria. |
tests/unit/test_pv_metadata_client.py |
Added coverage for key-only searches and optional criteria. |
tests/unit/test_machine_config_client.py |
Added configuration query coverage. |
tests/unit/test_machine_config_activation_client.py |
Added activation query coverage. |
src/dp_python_lib/client/query_client.py |
Updated v2 attribute helpers. |
src/dp_python_lib/client/pv_metadata_client.py |
Added key-only searches and optional criteria. |
src/dp_python_lib/client/machine_config_client.py |
Updated configuration APIs and type aliases. |
plan/tickets/41/plan.md |
Documents optional criteria implementation. |
plan/tickets/40/plan.md |
Documents key-only search implementation. |
doc/cookbook/query.md |
Documents key-only searches. |
doc/cookbook/pv-metadata.md |
Documents browsing and existence searches; contains examples needing explicit empty keys. |
doc/cookbook/machine-configuration.md |
Documents optional criteria and attributes. |
doc/cookbook/conventions.md |
Updates shared API conventions. |
CLAUDE.md |
Records unified conventions; related helper docstrings remain stale. |
Review details
- Files reviewed: 14/14 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ot review) DataSetQuery.attributes() and AnnotationQuery.attributes() still described themselves as differing from the older helpers "which require values", with relaxing those listed as pending issue #40 -- stale as of the previous commit in this PR, and contradicting the invariant CLAUDE.md now records. Both now state that every attribute helper behaves this way, with #40 named as the back-port that got them there. Also names the `key` argument in the pv-metadata cookbook's closing note rather than writing the helpers as bare `C.attributes()` / `PV.attr()`. Copilot read those as calls that would raise TypeError; they are inline prose references, not a fenced block, so nothing executes and the snippet checker never saw them -- but spelling them without their argument next to a real `Q.attributes("", ["0.49"])` call does read ambiguously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
craigmcchesney
deleted the
feature/issue-40-41-query-helper-relaxations
branch
September 10, 2026 21:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Back-ports the two criteria conventions #6 introduced to the five older criterion helpers and six older query/iter methods, so every criteria-based client in the library now spells them the same way.
Two commits, each with its own plan under
plan/tickets/.#40 — key-only
attributes()searchThe protos document an empty
valueslist on everyAttributesCriterionas a key-only existence search. Five helpers made it unreachable:PvMetadataQuery.attributes,ConfigurationQuery.attributes,ConfigurationActivationQuery.attributes,PvQuery.attr,ConfigQuery.attr. They now takevalues: list[str] | None = None, matchingDataSetQuery/AnnotationQueryfrom #6 — all seven attribute helpers agree.The empty-key rejection stays, and matters more on the two v2 selectors: the server does not validate the key there, so a blank one would reach Mongo as an existence test on
"attributes."and silently match nothing.Worth doing rather than leaving as a documented workaround:
conventions.mdjustified the guard as refusing "a criterion that would silently match everything." True oftags([]), false here — a key-only attributes criterion narrows the result set. The rule was generalized onto the one criterion it does not describe.pv-metadata.mdcarried a whole section teaching users to drop through to the raw protobuf classes, a snippet needing# cookbook:no-mypybecause the escape hatch is not statically checkable. It is now a positive recipe, and the checker type-checks all 107 snippets with none skipped for that reason.Triage also found the guard was not upstream-driven: it arrived in
c101964(2026-07-14) applying one rule acrosspv_name/aliases/tags/attributesat once, and all five proto comments predate it.#41 — optional
criteriaon the query/iter methodsSix methods took
criteriaas a required positional argument, a shape dating from when the server rejected an empty list. It is now match-all, soiter_pv_metadata()with no arguments is the browse-all form.criteriastays the first positional parameter, so existing callers are unaffected.Also adds a
_ActivationCriterionTypeAlias— the fully-qualified name is 109 characters and overflows the line limit in an annotated parameter. UsingTypeAliasrather than a bare assignment keeps mypy happy in an annotation position, and net drops one pre-existing mypy error.Verification
Server behavior was verified in dp-service source rather than taken from the tickets, since both rest on a server-behavior claim:
MongoQueryFilterBuilder.attributeFilter(), which returnsFilters.exists("attributes.<key>")on an empty list — not an$in: []that would match nothing.MongoQueryFilterBuilderTest.java:114-139covers that path for bothemptyList()andnull. Validation checks a blank key and deliberately does not check values: theTAGSCRITERIONarm immediately above each attributes arm does reject empty values, so the omission is a decision.DEFAULT_QUERY_LIMIT = 100), which is why the docstrings point atiter_*for browsing.Both ticket bodies were corrected before implementation. Notably, #41 claimed the default page size is server-configurable — it is a hardcoded constant, so the client docs deliberately do not promise it is tunable — and over-attributed the change to dp-grpc PR #147, which covers only three of the five RPCs.
725 unit tests pass(719 baseline + 6 net), ruff lint and format clean, 107 cookbook snippets checked.Closes #40
Closes #41
🤖 Generated with Claude Code
https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn