Skip to content

Back-port the key-only attributes search and optional criteria to the older clients (#40, #41) - #47

Merged
craigmcchesney merged 3 commits into
mainfrom
feature/issue-40-41-query-helper-relaxations
Sep 10, 2026
Merged

Back-port the key-only attributes search and optional criteria to the older clients (#40, #41)#47
craigmcchesney merged 3 commits into
mainfrom
feature/issue-40-41-query-helper-relaxations

Conversation

@craigmcchesney

Copy link
Copy Markdown
Collaborator

Back-ports the two criteria conventions #6 introduced to the five older criterion helpers and six older query/iter methods, so every criteria-based client in the library now spells them the same way.

Two commits, each with its own plan under plan/tickets/.

#40 — key-only attributes() search

The protos document an empty values list on every AttributesCriterion as a key-only existence search. Five helpers made it unreachable: PvMetadataQuery.attributes, ConfigurationQuery.attributes, ConfigurationActivationQuery.attributes, PvQuery.attr, ConfigQuery.attr. They now take values: list[str] | None = None, matching DataSetQuery/AnnotationQuery from #6 — all seven attribute helpers agree.

The empty-key rejection stays, and matters more on the two v2 selectors: the server does not validate the key there, so a blank one would reach Mongo as an existence test on "attributes." and silently match nothing.

Worth doing rather than leaving as a documented workaround:

  • conventions.md justified the guard as refusing "a criterion that would silently match everything." True of tags([]), false here — a key-only attributes criterion narrows the result set. The rule was generalized onto the one criterion it does not describe.
  • pv-metadata.md carried a whole section teaching users to drop through to the raw protobuf classes, a snippet needing # cookbook:no-mypy because the escape hatch is not statically checkable. It is now a positive recipe, and the checker type-checks all 107 snippets with none skipped for that reason.

Triage also found the guard was not upstream-driven: it arrived in c101964 (2026-07-14) applying one rule across pv_name/aliases/tags/attributes at once, and all five proto comments predate it.

#41 — optional criteria on the query/iter methods

Six methods took criteria as a required positional argument, a shape dating from when the server rejected an empty list. It is now match-all, so iter_pv_metadata() with no arguments is the browse-all form. criteria stays the first positional parameter, so existing callers are unaffected.

Also adds a _ActivationCriterion TypeAlias — the fully-qualified name is 109 characters and overflows the line limit in an annotated parameter. Using TypeAlias rather than a bare assignment keeps mypy happy in an annotation position, and net drops one pre-existing mypy error.

Verification

Server behavior was verified in dp-service source rather than taken from the tickets, since both rest on a server-behavior claim:

Both ticket bodies were corrected before implementation. Notably, #41 claimed the default page size is server-configurable — it is a hardcoded constant, so the client docs deliberately do not promise it is tunable — and over-attributed the change to dp-grpc PR #147, which covers only three of the five RPCs.

725 unit tests pass (719 baseline + 6 net), ruff lint and format clean, 107 cookbook snippets checked.

Closes #40
Closes #41

🤖 Generated with Claude Code

https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn

craigmcchesney and others added 2 commits September 10, 2026 15:18
…n helpers (#40)

The protos document an empty `values` list on every AttributesCriterion as a
key-only / existence search: match every record possessing the key, whatever
its value.  dp-service implements exactly that -- four of the five criteria
route to MongoQueryFilterBuilder.attributeFilter(), which returns
Filters.exists("attributes.<key>") on an empty list rather than an $in: []
that would match nothing, and MongoQueryFilterBuilderTest covers that path.

Five helpers made it unreachable, each raising on empty values:
PvMetadataQuery.attributes, ConfigurationQuery.attributes,
ConfigurationActivationQuery.attributes, PvQuery.attr, ConfigQuery.attr.
They now take `values: list[str] | None = None`, matching the shape #6
already shipped on DataSetQuery/AnnotationQuery, so all seven attribute
helpers spell the concept the same way.

The empty-key rejection stays, and matters more on the two v2 selectors:
the server does not validate the key there, so a blank one would reach Mongo
as an existence test on "attributes." and silently match nothing.

Why this was worth doing rather than left as a documented workaround:

- conventions.md justified the guard as refusing "a criterion that would
  silently match everything".  True of tags([]) and pv_name(), but false
  here -- a key-only attributes criterion narrows the result set.  The rule
  was generalized onto the one criterion it does not describe.
- pv-metadata.md carried a whole section teaching users to drop through to
  the raw protobuf classes, a snippet needing `# cookbook:no-mypy` because
  the escape hatch is not statically checkable.  That section is now a
  positive recipe and the checker type-checks all 104 snippets, none skipped
  for this reason.

The guard was not an upstream-driven decision: it arrived in c101964
(2026-07-14) applying one rule across pv_name/aliases/tags/attributes at
once, and all five proto comments predate it.

Five existing tests asserted the rejection and are inverted rather than
deleted, each covering both spellings (`attributes("k")` and
`attributes("k", [])`).  721 unit tests pass.

Plan: plan/tickets/40/plan.md.  Verified against dp-grpc 6dfff3f and
dp-service fddf692.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
…hods (#41)

Six methods took `criteria` as a required positional argument, a shape
dating from when the server rejected an empty criteria list.  The server
now treats an empty list as match-all, so "browse everything, paged" is a
legitimate call -- but from Python it read `iter_configurations([])`, which
looks like a mistake at the call site and was documented nowhere.

`criteria` is now optional on query_pv_metadata / iter_pv_metadata,
query_configurations / iter_configurations, and
query_configuration_activations / iter_configuration_activations, plus
their three private request builders, matching the shape #6 shipped on
datasets and annotations.  `iter_pv_metadata()` is the browse-all form.

Non-breaking: criteria remains the first positional parameter.

Server behavior verified rather than taken from the ticket.  Each
validation site carries a deliberate comment, not merely a missing check
(QueryPvMetadataJob.java:38 and the two siblings):

    // An empty criteria list is match-all by contract (#245), not an
    // error, so there is deliberately no list-level emptiness check here.

Per-criterion validation is retained everywhere, so this loosens the list
and not its contents.  Paging still applies: DEFAULT_QUERY_LIMIT = 100 in
MongoSyncAnnotationClient, applied unconditionally -- dropping the last
criterion does not change the page size -- which is why the docstrings
point at iter_* rather than a bare query_* for browsing.

Two corrections to the ticket body, recorded in the plan:

- The default page size is NOT configurable.  It is a hardcoded private
  static final int with no config key; the proto's "server-configured
  default page size" wording is loose and is deliberately not repeated in
  the client docs.
- dp-grpc #245 / PR #147 covers only the three metadata queries.  Datasets
  and annotations got match-all from the earlier #132 (7b2ea35) plus
  dp-service #248.  The net behavior is as the ticket says; only the
  attribution was wrong.

The v2 query methods are deliberately excluded: QueryParams still requires
a PV selector or config criteria, since a time-series query with no
selection is unbounded rather than a browse-all.

Also adds a _ActivationCriterion TypeAlias -- the fully-qualified name is
109 characters and overflows the line limit in an annotated parameter.
Using TypeAlias rather than a bare assignment keeps mypy happy in an
annotation position, and net drops one pre-existing mypy error.

725 unit tests pass; ruff clean; 107 cookbook snippets checked.

Plan: plan/tickets/41/plan.md.  Verified against dp-grpc 6dfff3f and
dp-service fddf692.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
Copilot AI lite review requested due to automatic review settings September 10, 2026 21:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two documentation nits remain: stale helper docstrings and invalid blank-key examples.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Back-ports key-only attribute searches and optional criteria to older client APIs.

Changes:

  • Added key-only attribute searches.
  • Made query criteria optional for match-all browsing.
  • Added tests and updated documentation, plans, and conventions.
File summaries
File Reviewed changes
tests/unit/test_query_client.py Added coverage for key-only attribute criteria.
tests/unit/test_pv_metadata_client.py Added coverage for key-only searches and optional criteria.
tests/unit/test_machine_config_client.py Added configuration query coverage.
tests/unit/test_machine_config_activation_client.py Added activation query coverage.
src/dp_python_lib/client/query_client.py Updated v2 attribute helpers.
src/dp_python_lib/client/pv_metadata_client.py Added key-only searches and optional criteria.
src/dp_python_lib/client/machine_config_client.py Updated configuration APIs and type aliases.
plan/tickets/41/plan.md Documents optional criteria implementation.
plan/tickets/40/plan.md Documents key-only search implementation.
doc/cookbook/query.md Documents key-only searches.
doc/cookbook/pv-metadata.md Documents browsing and existence searches; contains examples needing explicit empty keys.
doc/cookbook/machine-configuration.md Documents optional criteria and attributes.
doc/cookbook/conventions.md Updates shared API conventions.
CLAUDE.md Records unified conventions; related helper docstrings remain stale.
Review details
  • Files reviewed: 14/14 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread CLAUDE.md
Comment thread doc/cookbook/pv-metadata.md Outdated
…ot review)

DataSetQuery.attributes() and AnnotationQuery.attributes() still described
themselves as differing from the older helpers "which require values", with
relaxing those listed as pending issue #40 -- stale as of the previous
commit in this PR, and contradicting the invariant CLAUDE.md now records.
Both now state that every attribute helper behaves this way, with #40 named
as the back-port that got them there.

Also names the `key` argument in the pv-metadata cookbook's closing note
rather than writing the helpers as bare `C.attributes()` / `PV.attr()`.
Copilot read those as calls that would raise TypeError; they are inline
prose references, not a fenced block, so nothing executes and the snippet
checker never saw them -- but spelling them without their argument next to
a real `Q.attributes("", ["0.49"])` call does read ambiguously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019he3UCsAnqTDE2VQ73Djwn
@craigmcchesney
craigmcchesney merged commit 7824235 into main Sep 10, 2026
6 checks passed
@craigmcchesney
craigmcchesney deleted the feature/issue-40-41-query-helper-relaxations branch September 10, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants