Skip to content

Improve scenario prompt error messages HUD-522 - #249

Merged
lorenss-m merged 1 commit into
mainfrom
fix/scenario-prompt-error-message
Dec 31, 2025
Merged

lorenss-m merged 1 commit into
mainfrom
fix/scenario-prompt-error-message

Conversation

@ryantzr1

@ryantzr1 ryantzr1 commented Dec 21, 2025

Copy link
Copy Markdown
Contributor
  • Add detailed error when scenario not found with available options
  • Distinguish between missing scenario vs missing prompt setup
  • Show requested vs looked-up scenario IDs for debugging

Note

Enhances scenario prompt error handling/validation, clarifies agent errors for missing prompts, and adds robust cleanup on eval context setup failure.

  • Scenarios (hud/environment/scenarios.py):
    • More descriptive errors when remote scenario lookup fails, including requested vs looked-up ID and a list of available scenario IDs.
    • Validate get_prompt responses: raise on empty messages, malformed content, or non-string content.
  • Agent (hud/agents/base.py):
    • Distinguish between no scenario vs scenario returning empty prompt when ctx.prompt is missing, with targeted error messages.
  • Eval Context (hud/eval/context.py):
    • Wrap setup in __aenter__ with cleanup on failure, resetting context vars and disconnecting the environment before re-raising.

Written by Cursor Bugbot for commit 013eeae. This will update automatically on new commits. Configure here.

@ryantzr1 ryantzr1 changed the title Improve scenario prompt error messages Improve scenario prompt error messages HUD-522 Dec 21, 2025

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/environment/scenarios.py
@ryantzr1
ryantzr1 marked this pull request as draft December 21, 2025 06:12
Comment thread hud/agents/base.py
@ryantzr1
ryantzr1 force-pushed the fix/scenario-prompt-error-message branch 2 times, most recently from 0d41a10 to 3d3733c Compare December 21, 2025 06:50
@ryantzr1
ryantzr1 marked this pull request as ready for review December 21, 2025 06:51

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/environment/scenarios.py
@ryantzr1
ryantzr1 marked this pull request as draft December 21, 2025 06:58
- Add context-aware errors in MCPAgent when ctx.prompt is not set
- Show available scenarios when requested scenario is not found
- Handle malformed and empty scenario responses with clear messages
@ryantzr1
ryantzr1 force-pushed the fix/scenario-prompt-error-message branch from 3d3733c to 013eeae Compare December 21, 2025 07:02
@ryantzr1
ryantzr1 marked this pull request as ready for review December 21, 2025 07:03
@ryantzr1
ryantzr1 requested a review from jdchawla29 December 21, 2025 07:21
@lorenss-m
lorenss-m merged commit 8500d38 into main Dec 31, 2025
10 of 11 checks passed

This branch had an error being deployed

1 failed deployment
pre-release 013eeae3 Deployed Dec 21, 2025 by ryantzr1 via Run Evaluation Tests #8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants