MCP spec conformance: 5 requirements violated (via @hasmcp/mcp-spec-test) — spec 2025-11-25

[hasmcp-dev]

Running the black-box @hasmcp/mcp-spec-test conformance suite against the weather-server-python example in this repo, this time pinned to the older spec revision (2025-11-25) that the server also negotiates, 5 conformance checks fail. The core issue is that tools/list returns a JSON-RPC "Handler returned an invalid result" error, which in turn breaks pagination (nextCursor), the schema check on CallToolResult, and a version-less request; an official-SDK client also fails to list tools against this server. This is a separate report from the one filed for the 2026-07-28 revision, since the failure set is materially different under this revision. It may be an example-server bug in how tools are serialized, or an environment quirk of the test container — happy to have this closed if it's a false positive.

Conformance report

MCP 2025-11-25 conformance report

Verdict: not conformant — 5 requirements violated.

Target	sh -c '\[ -d /tmp/qr \] || git clone --depth 1 https://github.com/modelcontextprotocol/quickstart-resources /tmp/qr; cd /tmp/qr/weather-server-python && uv run weather.py'
Transport	stdio
Revision tested	2025-11-25
Revisions supported	2026-07-28, 2025-11-25
Passed	13
Failed	5
Not verified	2
Recommended, not met	1
Cases applied	20
Duration	8434ms
Generated	2026-08-24T09:29:48.223Z
Tool	@hasmcp/mcp-spec-test 0.1.1

Failed (5)

the server deviates from the spec here

Capability methods

• tools/list returns schema-conformant tools
  tools/list unexpected JSON-RPC error: {"code":-32603,"message":"Handler returned an invalid result"}
• tools/call returns a schema-conformant CallToolResult
  tools/list unexpected JSON-RPC error: {"code":-32603,"message":"Handler returned an invalid result"}
• following nextCursor terminates and does not repeat a page
  page 1: a cursor the server issued was refused: {"code":-32603,"message":"Handler returned an invalid result"}

Version negotiation

• a request with no version at all is served on the default
  a version-less tools/list must be served, not refused: {"code":-32603,"message":"Handler returned an invalid result"}

Official SDK interop

• a stock official-SDK client can list tools
  MCP error -32603: Handler returned an invalid result

Not verified (2)

skipped; a skip is not a pass

Capability methods

• prompts/get returns messages with a role and content
  target advertises the prompts capability but lists no prompts
• resources/read returns contents for every sampled resource
  target lists no resources to read

Recommended, not met (1)

the spec says SHOULD here, so this is not a conformance failure

Capability methods

• an invalid cursor is rejected with the wrong code
  the spec recommends -32602 (Invalid params); got -32603 (Handler returned an invalid result)

Passed (13)

checked and conformant

Capability methods

• tools/call on an unknown tool is an error, not a crash
• prompts/list returns schema-conformant prompts
• resources/list returns schema-conformant resources
• resources/templates/list returns schema-conformant templates
• resources/read on an unknown uri is an error
• an invalid pagination cursor is rejected (SHOULD)

server/discover

• the suite is reading a schema that matches the features it selected

Version negotiation

• initialize returns the schema-required fields
• the handshake settles on the revision under test
• an unsupported version offered at the handshake is refused or downgraded, not echoed

Official SDK interop

• the official SDK does not yet implement the newest revision
• a stock official-SDK client completes the handshake
• the handshake settles on a revision inside the supported window

How to reproduce

With Docker:

docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test -c "sh -c '[ -d /tmp/qr ] || git clone --depth 1 https://github.com/modelcontextprotocol/quickstart-resources /tmp/qr; cd /tmp/qr/weather-server-python && uv run weather.py'" --spec-version 2025-11-25

Without Docker (npx):

npx @hasmcp/mcp-spec-test@latest -c "sh -c '[ -d /tmp/qr ] || git clone --depth 1 https://github.com/modelcontextprotocol/quickstart-resources /tmp/qr; cd /tmp/qr/weather-server-python && uv run weather.py'" --spec-version 2025-11-25

---

This issue was generated by an automated MCP spec conformance sweep across public MCP server repos. If this doesn't apply here (e.g. an environment artifact of the test container), feel free to close it.