fix: surface HITL rejections as user decisions, not tool failures - #175
Conversation
A rejected HITL-gated tool call was reported to the model as a tool failure, causing it to invent technical reasons (e.g. insufficient stock) instead of acknowledging the user's choice. - Inject the decision note via wrapModelCall, the only hook that still runs after LangChain's jumpTo:"model" on reject/edit (beforeModel is skipped); clear the one-shot note via beforeModel on the next turn. - Add a reject branch to composeHitlDecisionNote so a rejection produces a note instead of returning undefined.
Akatuoro
left a comment
There was a problem hiding this comment.
Thanks for this fix @hm23 , much better result.
The model did already have the information that the user rejected the tool call... but it was only in the tool response, having it in a user message seems to yield better results.
What the LLM sees before the change
<|tool_call>call:submitOrder{book:201,quantity:1}<tool_call|>
<|tool_response>
response:submitOrder{value:<|"|>The user rejected this particular tool invocation with the reason: reject<|"|>}
<tool_response|>
<|tool_call>call:query{cql:<|"|>SELECT stock FROM Books WHERE ID = 201<|"|>}<tool_call|>
<|tool_response>
response:query{value:<|"|>count: 1
data[1]{stock}:
8<|"|>}
<tool_response|>
<|channel>thought
===== ASSISTANT =====
The user wants to order 1 copy of "Sturmhöhe".
I have already found that the book "Sturmhöhe" has ID 201 and currently has a stock of 8.
I attempted to call `submitOrder` with `book: 201` and `quantity: 1`, but the tool returned a rejection (simulated).
Wait, looking at the tool output: `The user rejected this particular tool invocation with the reason: reject`. This is an unusual response for a tool, usually it's an error from the service. If the tool call itself was "rejected" by the system/user in the simulation, I should inform the user.
However, let me check if there's any other way to fulfill the request or if I should just report the failure.
Actually, the `submitOrder` tool is what I used. The response `The user rejected this particular tool invocation with the reason: reject` seems to be a signal that the action could not be completed.
Let me try to check if there are any other books or if there's anything else I can do.
The user's request was "order sturmhöhe 1 copy".
I found the book ID (201) and the stock (8).
The order attempt failed.
I will inform the user that I found the book "Sturmhöhe" (ID: 201) but the order could not be processed.<channel|>I found the book "Sturmhöhe" (ID: 201) with a current stock of 8. However, I was unable to process your order for 1 copy at this time.
===== END =====
What the LLM sees after the change
<|tool_call>call:submitOrder{book:201,quantity:1}<tool_call|>
<|tool_response>
response:submitOrder{value:<|"|>The user rejected this particular tool invocation with the reason: reject<|"|>}
<tool_response|>
<turn|>
<|turn>user
User HITL decisions (not tool failures):
- User explicitly rejected `submitOrder({"book":201,"quantity":1})`.
<turn|>
<|turn>model
===== ASSISTANT =====
<|channel>thought
The user wants to order "sturmhöhe" 1 copy.
I previously identified "Sturmhöhe" with ID 201.
I attempted to call `submitOrder(book=201, quantity=1)`, but the user explicitly rejected it.
Since the user rejected the direct order, I should ask for clarification or confirmation before proceeding, or perhaps explain what happened. However, in a conversational context, if a user "rejects" a tool call in this simulated environment, it usually means I shouldn't have done it or there was an error in my logic (though here it's labeled as a HITL decision).
Actually, looking at the prompt history:
1. User: "order sturmhöhe 1 copy"
2. I: described Books, queried for 'sturmhöhe' (lowercase), didn't find it.
3. I: queried all Books, found "Sturmhöhe" with ID 201.
4. I: called `submitOrder(book=201, quantity=1)`.
5. System/User: "The user rejected this particular tool invocation with the reason: reject"
Wait, the user *rejected* the tool call. I should apologize or ask how they would like to proceed. Maybe they wanted to confirm something first? Or maybe I should have asked "Did you mean 'Sturmhöhe'?" before ordering.
Wait, usually, if a tool is rejected by HITL, I should acknowledge it.
Let me see if there's any other way to interpret "order sturmhöhe 1 copy".
Maybe I should check if there are other books or if I made a mistake in the ID.
The ID 201 is correct for "Sturmhöhe".
Perhaps the user wants me to confirm the details first?
"I found 'Sturmhöhe' (ID 201). Would you like me to proceed with ordering 1 copy?"
But I already tried to order.
Let's try to provide information about the book first to ensure it's the right one, or just ask for confirmation.
Actually, the user's prompt was a command.
Since the order was rejected, I'll ask for confirmation or if they want to see details first.
Wait, I'll check if there are any other details I can provide.
I'll try to ask the user if they want to see the details of "Sturmhöhe" before ordering.
Actually, I'll just respond saying I couldn't complete the order and ask how they want to proceed.<channel|>I attempted to place an order for "Sturmhöhe" (ID 201), but the order was not completed. Would you like me to show you the details of the book first, or is there something else you'd like to do?
===== END =====
SummaryThe following content is AI-generated and provides a summary of the pull request: TitleFix HITL rejection notes so models treat them as user decisions Category🐛 Bug Fix SummaryThis PR fixes how Human-in-the-Loop (HITL) tool rejections are communicated back to the model. Previously, rejected tool calls could be interpreted as technical tool failures, causing the model to speculate about incorrect failure reasons instead of recognizing that the user intentionally rejected the action. Changes
ImpactRejected HITL actions are now framed to the model as deliberate user decisions rather than failed tool executions, reducing misleading responses and preventing invented technical explanations. Test Plan
Have you...
Related: #102
PR Bot InformationVersion:
|
…for reject The previous commit moved all HITL note injection to wrapModelCall, which disturbed the edit path: LangChain only sets jumpTo:"model" for reject, so beforeModel still runs for edit and was already injecting the note correctly. Restore beforeModel inject-and-clear (handles edit) and keep wrapModelCall as a fallback for reject only (where beforeModel is skipped). Fixes the regressed hybrid eval "edit decision injects awareness note".
Summary
A deliberate user rejection of a HITL-gated tool call is currently surfaced to the model as a tool failure, causing it to invent technical reasons for the rejection (e.g. "the order could not be placed, possibly due to insufficient stock") instead of acknowledging that the user chose not to proceed.
This happens for two independent reasons:
hitlDecisionNoteInjectorMiddlewareruns in abeforeModelhook. LangChain'shumanInTheLoopMiddlewarereturnsjumpTo: "model"for reject/edit decisions, which re-enters the model node directly and skips allbeforeModelhooks. So even the existing edit-note was not reaching the model after a resume.composeHitlDecisionNotehas no branch forreject. It only produced a note foredit, so a rejection yieldedundefinedand nothing was injected.The reject signal that does reach the model is LangChain's synthetic
ToolMessage, but it carriesstatus: "error"(hard-coded in LangChain), which the model reads as a technical failure. Rejection was originally intended to be communicated purely via that ToolMessage content (see #102), but theerrorstatus defeats the content: the model treats the user's choice as something that went wrong.Approach
Use the existing decision-note mechanism — whose header already reads "User HITL decisions (not tool failures)" — as the in-CAP lever to correct the framing, and make it actually fire on the reject path:
beforeModeltowrapModelCall, the only hook that still runs after HITL'sjumpTo: "model". The middleware'sstateSchemamakes_hitlDecisionNoteavailable onrequest.statethere.beforeModelhook that clears the one-shot_hitlDecisionNoteon the next normal turn (wrapModelCallcannot update graph state), so the note is not re-injected on subsequent model calls.rejectbranch tocomposeHitlDecisionNoteso a rejection produces a note (- User explicitly rejected \submitOrder({...})`.`).This leaves LangChain untouched and keeps the ToolMessage channel as-is; the human-voice note after the tool result overrides the misleading
errorframing.Changes
lib/agents/middleware/hitl-decision-note-injector.js— inject viawrapModelCall; clear viabeforeModelsrv/handlers/graph-executor/hitl.js— addrejectbranch tocomposeHitlDecisionNotetests/integration/graph-executor-unit.test.js— update the previously edit-only expectation to assert a reject noteCHANGELOG.md— Fixed entryTest plan
npm test(vitest) green, in particulartests/integration/graph-executor-unit.test.js@agent.hitl, reject it in the preview; the model acknowledges the deliberate rejection and does not speculate about technical causes