feat(graders): add ASTCodeGrader and is_valid_python static code eval…

[shobhitagnihotri69]

Summary

Adds ASTCodeGrader and is_valid_python helper to hud.graders, enabling fast, zero-execution Python AST code evaluation and anti-cheat validation for coding agent rollouts and evaluation benchmarks.

Problem / Motivation

Evaluating generated code in agent rollouts currently relies on BashGrader (running shell commands) or LLM judges.
In high-throughput coding agent benchmarks and RL rollouts (e.g. HumanEval, code synthesis, competitive programming):

1. Invoking shell subprocesses for simple syntactic correctness checks is slow and incurs execution overhead.
2. Executing untrusted code directly in bash poses sandbox escape and security risks unless heavily isolated.
3. Benchmarks frequently need anti-cheat validation (e.g., verifying an agent didn't import forbidden libraries like subprocess or os, or execute dangerous calls like eval/exec).

Solution

1. ASTCodeGrader (Async Grader):
   • Subclasses Grader and integrates natively into combine(...).
   • Parses code via Python standard library ast.parse without executing code.
   • Supports definition verification (required_functions, required_classes).
   • Supports anti-cheat security policies (disallowed_imports, disallowed_calls).
   • Returns structured diagnostics in SubScore(info={...}) (including extracted functions, classes, imports, and violation lists).
2. is_valid_python(code) (One-liner helper):
   • Returns 1.0 if valid syntax, else 0.0.
3. Exported in hud.graders:
   • Full parity with existing graders (BashGrader, LLMJudgeGrader).

Test Coverage

• Added TestASTCodeGrader to hud/tests/test_graders.py:
  • test_is_valid_python_helper
  • test_ast_grader_valid_code
  • test_ast_grader_syntax_error
  • test_ast_grader_required_definitions
  • test_ast_grader_anti_cheat_disallowed_imports
  • test_ast_grader_anti_cheat_disallowed_calls
  • test_ast_grader_grade_in_combine
• All 82 tests pass (82 passed in 1.31s).
• All ruff checks pass cleanly.

---

Note

Low Risk
Additive grader module with no execution of agent code; only standard-library AST parsing and new tests, with no changes to existing grading paths.

Overview
Adds static Python code grading to hud.graders so benchmarks can check generated code without shelling out or executing untrusted submissions.

ASTCodeGrader parses code with ast.parse and returns a binary SubScore, optionally enforcing required function/class names and anti-cheat rules (disallowed_imports, disallowed_calls). Diagnostics land in SubScore.info (syntax errors, defined symbols, imports, violations). It follows the existing Grader / combine pattern via ASTCodeGrader.grade(...).

is_valid_python is a small helper that returns 1.0 / 0.0 for syntactic validity only.

Both symbols are re-exported from hud.graders. TestASTCodeGrader covers syntax, required definitions, import/call policies, and composition with combine.

^{Reviewed by Cursor Bugbot for commit 66b400e. Bugbot is set up for automated code reviews on this repo. Configure here.}