[sdk] Native support for LangChain/LangGraph, CrewAI, LlamaIndex and Pydantic AI, on a real identity layer by SiddarthAA · Pull Request #730 · FailproofAI/failproofai

SiddarthAA · 2026-08-19T14:44:49Z

What

The Python SDK was a capture surface you had to operate by hand: 15 keyword-only
emit methods, every one requiring session_id= and agent_id=, and nothing
propagating them. There was no ambient session anywhere in the package — no
decorator, no context manager, no contextvars. skill/SKILL.md stated that as a
deliberate contract, and skill/references/integration.md shipped a ~60-line
contextvars wrapper as markdown for customers to paste into their own
codebase: the SDK asking its users to write the missing half of the SDK.

So every customer on LangGraph, CrewAI or LlamaIndex hand-wrote the same
adapter, got thread propagation wrong, and abandoned it halfway — the exact
failure that reference doc opens by describing.

Now, with the call site unchanged:

import failproofai_sdk

failproofai_sdk.configure(environment="prod")
failproofai_sdk.instrument()          # auto-detects frameworks already imported

graph.invoke({"messages": [...]})     # unchanged

Ported from FailproofAI/agenteye#503,
reconciled against the eleven SDK bugs fixed on feat/fp-cli since that branch
was cut. Ported rather than copied: #503's _writer.py is 80 lines against
our ~450, so a straight copy would have silently reverted every one of them.

Why this is not the same as emitting events by hand

Measured, not asserted. The same task, same model, same tool, instrumented two
ways:

	manual (~8 lines of event calls)	`instrument()` (1 line)
events	4	14
event types	4	8
tool events	0	2
durations	0	6
token counts	0	2

The manual version reported one model_request/model_response pair. The
agent made two LLM calls. It recorded zero tool events for a run whose
entire point was calling a tool.

That is not carelessness — it is the ceiling of the approach. graph.invoke()
is one call seen from outside; the ReAct loop, the tool dispatch, the second
round-trip and the per-node timings all happen inside it. You cannot instrument
what you cannot see, and you can only emit what you remember to emit.

What is here

Ambient identity (_context.py, _scopes.py, _runtime.py) — session(),
agent() and tool_call() under both with and async with, current(), and
propagate() for thread hand-off. session_id/agent_id became optional on
all 15 methods, falling back to context, without breaking a single existing
call — which is why the golden wire-format bytes are untouched.

Four adapters — LangChain/LangGraph, CrewAI, LlamaIndex, Pydantic AI. Each
is a translation table over one shared RunTracker, emitting only the existing
15 event types, so nothing fans out to the server, collector, CLI or the stored
schema.

AutoGen is deliberately absent: autogen-core 0.7.5 last shipped 2025-09-30
with no commits since, and Microsoft's forward path is a separate package. The
live product is AG2, a different distribution whose middleware has no global
auto-instrument hook.

request_id on the two model events. The dashboard pairs model events on
it, but no SDK method accepted one and no doc mentioned it — so every
integration written to our own documentation emitted unpairable model events,
including demo-agent/mock_agent.py.

Three decisions where #503 was right and the current branch was wrong

Pairing keys are session-scoped, never agent-scoped. feat/fp-cli had
just tightened _pending keys to kind:session:agent:id to stop two
concurrent sessions colliding. That over-tightened: once a framework runs
tools inside sub-agents, a tool_use opened under planner and closed under
worker is the ORDINARY case, and an agent-scoped key makes those pairs miss
entirely — silently dropping duration_ms for exactly the nested runs that
most need it. The rule that survives both: include what makes the id unique
(kind, session), exclude what can legitimately change between the two events
(the agent).
TypeError, not ValueError, for missing or mistyped identity. It is
what a caller got before identity became optional, so code catching one keeps
working.
Field validation runs before identity resolution. A reserved **field is
a fault in the call itself and reads the same from anywhere; the identity
error depends on where the call was made from.

One decision where the current branch was right and #503 was not

#503 fixes the _track_pending KeyError — the same race found independently
here — with a threading.Lock. That lock is kept out: a lock held at the
instant of a fork() is inherited locked by a thread that does not exist in the
child, which is the exact hazard _writer rebuilds its Event and lock to
avoid. The tolerant, lock-free eviction on this branch fixes the same crash with
no fork edge.

Zero dependencies survives, and the test got stronger

The adapters import the frameworks they adapt — there is no other way to
subclass a callback base class. But integrations/__init__ resolves them by
string through importlib.import_module at call time, so:

the source scan is scoped to core modules with a per-file allowlist, and a new
file under integrations/ is scanned like core code until it is named there;
the promise is now asserted at runtime: a fresh interpreter imports the
package and must have no framework in sys.modules;
[project.dependencies] is still empty, and CI still installs the built wheel
with --no-deps.

An eager adapter import is not a style problem — it makes import failproofai_sdk raise ImportError on every machine without that framework.
Verified by planting one.

Framework extras carry upper bounds, because without one a clean build a year
from now pulls the next major, the callback API shifts, and the adapter stops
receiving events while raising nothing. There is deliberately no [all].

Verification

Not "the tests pass" — the couplings here fail silently, so each layer was
checked against something that could disagree.

Layer	Result
SDK suite	644 pass without frameworks, 855 with them (was 429)
Adapter suites, real frameworks	langchain 50, crewai 49, llama_index 44, pydantic_ai 68
Every commit, independently	426 → 503 → 503 → 617 → 638 → 640, by clean extraction
Python matrix	3.10 / 3.11 / 3.12 / 3.13 / 3.14, contract tests enforced
Live end-to-end	all four frameworks → daemon → dashboard `/v1/events` → events store

The live run used a real model. Captured and confirmed in the store:

lab-langgraph    44 events   5 agents   14 types
lab-llamaindex   28 events   2 agents    8 types
lab-pydantic-ai  16 events   4 agents    6 types
lab-crewai       14 events   3 agents    6 types

All 15 event types were confirmed end to end, with promoted columns
populated (tool_name, model, duration_ms, tokens), the framework field
on every event, and parent_id nesting intact. 18 batches uploaded, 0
failed.

Beyond the happy path, per framework: a tool that raises is recorded on
tool_result (4/4), the async path is captured (3/3 where async applies), and
concurrent sessions across threads stay isolated.

The four quickstarts in examples/ were executed against a real model
before being committed; parsing is not evidence that an example works.

Not checked

Streaming (.astream), CrewAI flows, LlamaIndex workflows beyond
FunctionAgent, provider retry/rate-limit paths, long-run memory behaviour, and
uninstrument() round-trips under load.

For the reviewer

FailproofAI is now public API that users type — capabilities=[FailproofAI()],
FailproofAITracer, FailproofAICrewListener. Renamed from #503's AgentEye*.
Worth an explicit yes on the naming before it ships.

AGENTEYE_HOME, AGENTEYE_ENVIRONMENT and ~/.agenteye are untouched — they
are a contract with two separately-released daemons. AGENTEYE_STRICT was new
in #503 and nothing else reads it, so it became FAILPROOFAI_SDK_STRICT.

#503 also carried two dashboard fixes (executionGraph.ts, sessionSummary.ts).
Those live in the AgentEye repo and are not in this PR; they need a
companion change there.

Hermes review

Field	Value
Status	Changes requested
Reviewed commit	`6047ade3dae5bd4157621045ced8792430da83f4`
Policy revision	`1d8f31d926828f3bae215c58f5b35baa44acbff0`
Model	`gpt-5.6-terra`
Duration	398s
Updated	2026-08-19T14:57:34.863547764+00:00

Summary

Found a high-confidence Pydantic AI integration failure: the documented explicit capability is inert unless auto-instrumentation was also enabled. Also found invalid falsy session IDs silently generating a new session. Syntax compilation passed; the full Python suite could not be provisioned in the isolated container.

Changes

Added ambient session, agent, and tool-call scopes with context propagation.
Made event identity optional and added model request IDs.
Added lazy adapters for LangChain/LangGraph, CrewAI, LlamaIndex, and Pydantic AI.
Added framework extras, examples, docs, and adapter tests.

Validation

Passed docker run --rm -v /review/input/workspace:/workspace:ro -w /workspace/sdk/python python:3.12-slim python -c '<compile all SDK and test Python sources>' — All Python source files under failproofai_sdk and tests compiled successfully. (10s)
Skipped docker run --rm -v /review/input/workspace:/workspace:ro -w /workspace/sdk/python python:3.12-slim sh -lc 'python -m pip install .[dev] && pytest -q -m "not framework"' — The isolated container could not provision pytest because its package-index request produced no response; this external dependency is unavailable to the harness. (60s)

Findings

High/High Explicit Pydantic AI capabilities are inert unless auto-instrumentation is also enabled — The adapter documents capabilities=[FailproofAI()] as a standalone usage path (lines 345-353), but all three capability wrappers immediately return the underlying handler while module-global _enabled is false (for example lines 394-396). _enabled starts false and is only set true by _Adapter.install() (lines 671-684). Thus a user following the explicit-capability API without first calling instrument("pydantic_ai") gets no telemetry at all. The explicit-capability test runs under the instrumented fixture, so it cannot expose this path. (sdk/python/failproofai_sdk/integrations/pydantic_ai.py:394)

1 advisory finding

Medium/High Falsy invalid session IDs are converted into unrelated generated sessions — session._enter() selects the requested ID with self._requested or ... (line 97), and agent._enter() does the same (line 197). Consequently session("") and agent(..., session_id=0) do not raise identity validation errors; they generate UUID sessions instead. A containerized reproduction printed generated UUIDs for both cases. This silently splits telemetry from the caller's intended session rather than preserving the event API's invalid-identity failure behavior. (sdk/python/failproofai_sdk/_scopes.py:97)

Open questions

None.

Policy overrides

None.

`_pending` correlates a start event with its end so `duration_ms` can be measured. The key it uses has been wrong twice, in opposite directions. Bare ids were the first mistake: tool pairs keyed on `tool_call_id` and hook pairs on `hook_id` shared one flat keyspace, so a caller whose tool call and hook happened to share an id — not exotic, both are frequently the harness's own step id — got a `hook_completed` that consumed the `tool_use` timestamp and then a `tool_result` with no duration at all. Adding the session fixed a second, real collision: `_pending` lives on one process-wide namespace, so two concurrent sessions collided on any shared step id. Starting `step-1` in session A and then in B overwrote A's timestamp; A's result reported B's interval and B's reported none. Adding the AGENT as well was over-tightening, and this commit removes it. Once a framework runs tools inside sub-agents — LangGraph and CrewAI both do — a `tool_use` opened under `planner` and closed under `worker` is the ORDINARY case, and an agent-scoped key makes those pairs miss entirely, silently dropping `duration_ms` for exactly the nested runs that most need it. The rule that survives both: include what makes the id unique (kind, session), exclude what can legitimately change between the two events (the agent). A session cannot change under a pair; an agent can. Applied to all four pair types, since a `human_wait` answered by a supervisor and an `agent_pause` resumed by another agent are the same shape. These are correlation keys only — never emitted, never leaving the process — so no wire format changes. Only `duration_ms` changes, in the colliding cases, from a fabricated or missing value to a correct one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Every event method took `session_id` and `agent_id` as required keyword arguments and nothing propagated them, so instrumenting a real agent meant threading two ids through every function that might emit. That is the diff nobody wants to review, and it is why `skill/references/integration.md` shipped a ~60-line contextvars wrapper AS MARKDOWN for customers to paste into their own codebase — the SDK asking users to write the missing half of the SDK. Three scopes bind identity on contextvars instead: with failproofai_sdk.session() as sid: with failproofai_sdk.agent("planner", goal=q): # agent_start/end with failproofai_sdk.tool_call("search") as t: # tool_use/result t.output = search(q) All three work under `with` and `async with` — an agent framework is half-async, and `@contextmanager` supports only the former, so these are plain classes whose async pair delegates to the sync pair. No scope awaits anything (`submit()` is a deque append), so the delegation is not a lie. `session_id`/`agent_id` are now OPTIONAL on all 15 methods, resolved from the scope when omitted. Existing call sites are untouched and still pass ids explicitly, which is why the golden wire-format bytes are unchanged. Details that are load-bearing rather than incidental: * The agent stack is a TUPLE. A `ContextVar[list]` is shared by reference across tasks and threads, so `.append()` in one mutates what every other sees — the cross-run mixing contextvars exist to prevent, wearing a contextvars costume. It passes every single-threaded test. * `propagate(fn)` snapshots VALUES rather than using `copy_context().run`. A `Context` cannot be entered twice, so the copy_context form crashes the caller's worker on any reuse — `pool.map`, a retried submit — and mutations inside `ctx.run` persist, leaking one call's agent stack into the next. * `agent()` emits `error` strictly BEFORE `agent_end`, because the dashboard closes the span at `agent_end` and anything after it is attributed to nothing. A cancellation closes as `cancelled`, not `failed` — a cancelled run is not an error, and marking it one pollutes the Errors surface. * Identity is validated AFTER resolution, never before. Validating first would reject every ambient call; resolving without validating would restore the silent skip, since ingest drops an event whose `session_id` is not a JSON string and answers `200 OK` with `{"accepted":0,"skipped":1}`. * Unresolvable identity raises TypeError, not ValueError — the wrong type, or a missing required argument, which is exactly what a caller got before this change. Code catching TypeError keeps working. * Field validation runs before identity resolution: a reserved `**field` is a fault in the call itself and reads the same from anywhere, so it gives a stable message; the identity error depends on where the call was made from. `conftest.py` gains the suite-wide isolation this makes necessary: a per-test spool, restored process globals, and an assertion that a test leaking a scope FAILS rather than quietly misattributing every event after it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The dashboard pairs a model request with its response on `request_id`, but no SDK method accepted one and no doc mentioned it. So every integration written to our own documentation emitted model events that cannot be paired — including `demo-agent/mock_agent.py`, our own reference implementation. Optional on both methods, and appended LAST in the ordered field list, so an event that omits it serialises byte-for-byte as before. That matters twice over: `test_wire_format.py` freezes those bytes, and ingest's dedup key hashes the canonical payload, so a reordering would stop retried batches collapsing and surface as duplicate rows rather than as an error. Only the two model events carry it. The other thirteen have nothing to pair with, and a field most event types cannot use is a field people fill in wrongly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…x, Pydantic AI failproofai_sdk.configure(environment="prod") failproofai_sdk.instrument() # auto-detects what is already imported graph.invoke({"messages": [...]}) # unchanged Each adapter is a translation table over one shared `RunTracker`, emitting only the existing 15 event types — nothing fans out to the server, collector, CLI or the stored schema. WHY THIS IS NOT THE SAME AS EMITTING BY HAND. Measured on one task, same model, same tool: hand-written instrumentation produced 4 events and 4 types; the adapter produced 14 events and 8 types. The manual version reported ONE model_request/model_response pair for a run that made TWO LLM calls, and zero tool events for a run whose entire point was calling a tool. That is not carelessness, it is the ceiling: `graph.invoke()` is one call from outside, and the ReAct loop, the tool dispatch, the second round-trip and the per-node timings all happen inside it. You cannot instrument what you cannot see. AutoGen is deliberately absent. `autogen-core` 0.7.5 last shipped 2025-09-30 with no commits since, and Microsoft's forward path is a separate package; the live product is AG2, a different distribution whose middleware has no global auto-instrument hook. ZERO DEPENDENCIES SURVIVES THIS, and the test got stronger rather than weaker. The adapters import the frameworks they adapt — there is no other way to subclass a callback base class — but `integrations/__init__` resolves them by STRING through `importlib.import_module` at call time. So the source scan is now scoped to core modules with a per-file allowlist, and the promise is asserted at runtime instead: a fresh interpreter imports the package and must have no framework in `sys.modules`. An eager import is not a style problem, it makes `import failproofai_sdk` raise ImportError on every machine without that framework — verified by planting one. Framework extras carry upper bounds. Without one, a clean build a year from now pulls the next major, the callback API shifts, and the adapter stops receiving events while raising nothing — an empty dashboard, not a traceback. There is deliberately no `[all]`: an extra installing four agent frameworks at once is a resolver problem handed to somebody who wanted a telemetry library. Verified against the real frameworks, not mocks: 211 adapter tests (langchain 50, crewai 49, llama_index 44, pydantic_ai 68), plus live runs of all four against a real model, each reaching the daemon and the events store. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`examples/*_quickstart.py` — about 30 lines each, the thing somebody runs in their first five minutes. All four were executed against a real model before being committed; parsing is not evidence that an example works. `tests/test_examples.py` guards them, because nothing else can: they need a framework and an API key, so they cannot run in unit CI, which is exactly why they rot. It checks they parse, that they call API this package actually exports, that each imports only the framework its own extra installs, that the extra they name exists — and that they demonstrate the ergonomics they exist to demonstrate. An example that threads `session_id=` by hand teaches the manual path the scopes were built to remove, so that fails the suite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`skill/SKILL.md` §3 opened with "There is no ambient session. No decorator, no context manager, no contextvar, no `set_session()`." An agent reads that as the contract and writes against it, so leaving it would have been worse than not documenting the scopes at all — a skill is instructions somebody executes. - `skill/references/frameworks.md` — new. Per-framework mapping tables, what every adapter guarantees, how to mix adapters with hand-written events, how to verify one, and what to do for a framework not on the list. - `README.md` — the frameworks and scopes sections, ahead of the manual event reference, because that is now the order people meet them in. - `skill/SKILL.md` — §3 rewritten to describe the scopes, and pointed at the new reference rather than at the wrapper customers used to paste in by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

github-actions · 2026-08-19T14:44:58Z

Thanks @SiddarthAA for your contribution to Failproof AI! 🙌

We'd love to discuss your PR and welcome you to our community: https://discord.befailproof.ai/

coderabbitai · 2026-08-19T14:45:04Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1c2e01f0-8dcc-4a8b-9c2b-1fd4095eb2be

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Every PR carries an entry, and this one touched nothing outside sdk/python until now — which is exactly how a release note goes missing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

hermes-exosphere · 2026-08-19T14:47:08Z

Hermes


Status	Reviewed
Verdict	Changes requested
Head	`6047ade3dae5`
Rounds	1 of 5

Found a high-confidence Pydantic AI integration failure: the documented explicit capability is inert unless auto-instrumentation was also enabled. Also found invalid falsy session IDs silently generating a new session. Syntax compilation passed; the full Python suite could not be provisioned in the isolated container.

What this changes

flowchart LR
    n0PythonSDKidentityscopes["+ Python SDK identity scopes"]
    n1EventemissionAPI["~ Event emission API"]
    n2Integrationregistry["+ Integration registry"]
    n3LangChainandLangGraphadapter["+ LangChain and LangGraph adapter"]
    n4CrewAIadapter["+ CrewAI adapter"]
    n5LlamaIndexadapter["+ LlamaIndex adapter"]
    n6PydanticAIadapter["+ Pydantic AI adapter"]
    n7SDKpackagingandguidance["~ SDK packaging and guidance"]
    n0PythonSDKidentityscopes -- "ambient identity" --> n1EventemissionAPI
    n2Integrationregistry -- "lazy loads" --> n3LangChainandLangGraphadapter
    n2Integrationregistry -- "lazy loads" --> n4CrewAIadapter
    n2Integrationregistry -- "lazy loads" --> n5LlamaIndexadapter
    n2Integrationregistry -- "lazy loads" --> n6PydanticAIadapter
    n3LangChainandLangGraphadapter -- "translated events" --> n1EventemissionAPI
    n4CrewAIadapter -- "translated events" --> n1EventemissionAPI
    n5LlamaIndexadapter -- "translated events" --> n1EventemissionAPI
    n6PydanticAIadapter -- "translated events" --> n1EventemissionAPI

Rounds

Round	Reviewed	Commits in this round	Verdict
1	`6047ade3dae5`	`ae0b61de0c94` `79abe63ae53c` `56473dce2101` `8e884637c7b9` `0bf8670f25b3` `6da96c33aca0` `69aa34e88bcf` `6047ade3dae5`	Changes requested — F1

Findings

Open

F1 Explicit Pydantic AI capabilities are inert unless auto-instrumentation is also enabled (sdk/python/failproofai_sdk/integrations/pydantic_ai.py) — round 1
F2 Falsy invalid session IDs are converted into unrelated generated sessions (sdk/python/failproofai_sdk/_scopes.py) — round 1

@hermes-exosphere help lists every command. This comment is maintained in place — I rewrite it after each review rather than posting a new one.

…ey replaced Three gaps found by diffing this branch against the upstream PR it was ported from (FailproofAI/agenteye#503), of which the first would have failed CI. **uv.lock was stale.** `pyproject` grew `pytest-asyncio` and five framework extras; the lockfile had none of them, so `uv sync --locked --extra dev` — the exact command the `failproofai-sdk` CI job runs — failed with "the lockfile needs to be updated". Regenerated: +5941/-94, which is most of the line-count difference between the two PRs and an omission rather than a saving. All five framework extras resolve, and `uv sync --locked --extra pydantic-ai` installs. **`integration.md` still shipped the wrapper.** Its "## The wrapper" section was ~60 lines of contextvars scaffolding for customers to paste into their own codebase — the thing `session()`/`agent()`/`tool_call()` now are. Worse than redundant: it taught `contextvars.copy_context().run` for thread hand-off, which `_context.propagate` documents as broken, because a `Context` cannot be entered by two threads at once and so the copy-context form crashes the caller's worker on any reuse — `pool.map`, a retried submit. That is now a "do not reach for this" warning next to `propagate()`. **`events.md`** picks up the scopes and the four `human_*` events alongside them. Both ported files re-introduced a bug this branch had already fixed: their lifecycle brackets catch `Exception`, and `asyncio.CancelledError` inherits from `BaseException`, so a cancelled tool emits `tool_use` with no `tool_result` and a cancelled run gets no `agent_end` at all. `tests/test_skill_snippets.py` caught both on the way in, which is the whole reason it parses every fenced block rather than trusting review. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

hermes-exosphere

Hermes found blocking issues that should be addressed.

Review coverage was incomplete, but the concrete blocking findings below are sufficient to request changes.

High: Explicit Pydantic AI capabilities are inert unless auto-instrumentation is also enabled

Rule: COR-001
Location: sdk/python/failproofai_sdk/integrations/pydantic_ai.py:394
Evidence: The adapter documents capabilities=[FailproofAI()] as a standalone usage path (lines 345-353), but all three capability wrappers immediately return the underlying handler while module-global _enabled is false (for example lines 394-396). _enabled starts false and is only set true by _Adapter.install() (lines 671-684). Thus a user following the explicit-capability API without first calling instrument("pydantic_ai") gets no telemetry at all. The explicit-capability test runs under the instrumented fixture, so it cannot expose this path.
Required change: Make explicit FailproofAI() instances active independently of registry installation; keep any uninstall disablement scoped to auto-injected instances. Add a test that constructs Agent(..., capabilities=[FailproofAI()]) without calling instrument() and asserts emitted events.

1 advisory finding

Medium/High Falsy invalid session IDs are converted into unrelated generated sessions — session._enter() selects the requested ID with self._requested or ... (line 97), and agent._enter() does the same (line 197). Consequently session("") and agent(..., session_id=0) do not raise identity validation errors; they generate UUID sessions instead. A containerized reproduction printed generated UUIDs for both cases. This silently splits telemetry from the caller's intended session rather than preserving the event API's invalid-identity failure behavior. (sdk/python/failproofai_sdk/_scopes.py:97)

hermes-exosphere · 2026-08-19T14:57:37Z

+
+    # -- run --------------------------------------------------------------
+
+    async def wrap_run(self, ctx, *, handler):


Hermes — High/High (COR-001): Explicit Pydantic AI capabilities are inert unless auto-instrumentation is also enabled

The adapter documents capabilities=[FailproofAI()] as a standalone usage path (lines 345-353), but all three capability wrappers immediately return the underlying handler while module-global _enabled is false (for example lines 394-396). _enabled starts false and is only set true by _Adapter.install() (lines 671-684). Thus a user following the explicit-capability API without first calling instrument("pydantic_ai") gets no telemetry at all. The explicit-capability test runs under the instrumented fixture, so it cannot expose this path.

Required change: Make explicit FailproofAI() instances active independently of registry installation; keep any uninstall disablement scoped to auto-injected instances. Add a test that constructs Agent(..., capabilities=[FailproofAI()]) without calling instrument() and asserts emitted events.

hermes-exosphere · 2026-08-19T14:57:38Z

+        self._agent_token: "contextvars.Token | None" = None
+
+    def _enter(self) -> str:
+        sid = self._requested or _context.session_id() or uuid.uuid4().hex


Hermes — Medium/High (COR-001): Falsy invalid session IDs are converted into unrelated generated sessions

session._enter() selects the requested ID with self._requested or ... (line 97), and agent._enter() does the same (line 197). Consequently session("") and agent(..., session_id=0) do not raise identity validation errors; they generate UUID sessions instead. A containerized reproduction printed generated UUIDs for both cases. This silently splits telemetry from the caller's intended session rather than preserving the event API's invalid-identity failure behavior.

Required change: Distinguish omission from a supplied value with is None, then validate supplied session IDs before binding them. Cover empty strings and falsy non-string values with unit tests.

SiddarthAA and others added 6 commits August 19, 2026 20:01

docs(changelog): record the SDK framework adapters (#730)

69aa34e

Every PR carries an entry, and this one touched nothing outside sdk/python until now — which is exactly how a release note goes missing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

hermes-exosphere requested changes Aug 19, 2026

View reviewed changes

Navigation Menu

Uh oh!

[sdk] Native support for LangChain/LangGraph, CrewAI, LlamaIndex and Pydantic AI, on a real identity layer - #730

[sdk] Native support for LangChain/LangGraph, CrewAI, LlamaIndex and Pydantic AI, on a real identity layer#730
SiddarthAA wants to merge 8 commits into
feat/fp-clifrom
sdk/framework-adapters

SiddarthAA commented Aug 19, 2026 •

edited by hermes-exosphere

Loading

Uh oh!

github-actions Bot commented Aug 19, 2026

Uh oh!

coderabbitai Bot commented Aug 19, 2026 •

edited

Loading

Review skipped

Uh oh!

hermes-exosphere commented Aug 19, 2026 •

edited

Loading

Uh oh!

hermes-exosphere left a comment

Uh oh!

hermes-exosphere Aug 19, 2026

Uh oh!

hermes-exosphere Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		# -- run --------------------------------------------------------------

		async def wrap_run(self, ctx, *, handler):

Uh oh!

Conversation

SiddarthAA commented Aug 19, 2026 • edited by hermes-exosphere Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why this is not the same as emitting events by hand

What is here

Three decisions where #503 was right and the current branch was wrong

One decision where the current branch was right and #503 was not

Zero dependencies survives, and the test got stronger

Verification

Not checked

For the reviewer

Hermes review

Summary

Changes

Validation

Findings

Open questions

Policy overrides

Uh oh!

github-actions Bot commented Aug 19, 2026

Uh oh!

coderabbitai Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

hermes-exosphere commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Hermes

What this changes

Rounds

Findings

Uh oh!

hermes-exosphere left a comment

Choose a reason for hiding this comment

High: Explicit Pydantic AI capabilities are inert unless auto-instrumentation is also enabled

Uh oh!

hermes-exosphere Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

hermes-exosphere Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SiddarthAA commented Aug 19, 2026 •

edited by hermes-exosphere

Loading

coderabbitai Bot commented Aug 19, 2026 •

edited

Loading

hermes-exosphere commented Aug 19, 2026 •

edited

Loading