Skip to content

Add TypeSafe / Jev System One pass for U-NSGA-III Pareto scoring - #5

Merged
jkbennitt merged 1 commit into
mainfrom
cursor/typesafe-pareto-score-b44f
Sep 16, 2026
Merged

jkbennitt merged 1 commit into
mainfrom
cursor/typesafe-pareto-score-b44f

Conversation

@jkbennitt

Copy link
Copy Markdown
Member

Summary

Adds an optional TypeSafe / Jev System One helper beside classical Pareto fronts so a small candidate sample can get calibrated Score and Choice judgments (constraint satisfaction, diversity, exploit-vs-explore, keep/drop/review). This is an additive semantic layer — it does not replace NSGA-III / U-NSGA-III objectives.

Python under tools/typesafe-pareto/ (same pattern as tools/oracle). One POST https://api.typesafe.ai/v1/systemone fans out all questions. Official typesafe-sdk is optional; stdlib HTTP matches the documented API. Live calls only when TYPESAFE_API_KEY is set. No keys in the repo.

Checklist

  • dotnet test Unsga3.slnx -c Release not required (library/algorithm unchanged)
  • CHANGELOG.md updated under Unreleased
  • Public API unchanged
  • Survival / normalization / metrics (IGD) untouched — no oracle impact

Test plan

  • python -m unittest tools/typesafe-pareto/test_score_pareto.py -v (16 mock tests)
  • python tools/typesafe-pareto/score_pareto.py --smoke --force-mock
  • New CI job typesafe-pareto (Python 3.12, no secrets)
  • Metrics append {ts, experiment:"unsga3_pareto_score", repo:"AppSprout-dev/Unsga3", model, latency_ms, usage, candidate_count, answers, notes} to metrics/typesafe-runs.jsonl (gitignored)

How to run

python tools/typesafe-pareto/score_pareto.py --smoke --force-mock
# live: export TYPESAFE_API_KEY=...  (never commit)

Docs used: HTTP API, Python SDK, fan-out.

Open in Web Open in Cursor 

Optional tools/typesafe-pareto helper fans out Score + Choice judgments
over a small U-NSGA-III front sample. Live calls only when
TYPESAFE_API_KEY is set; CI runs mocks and writes metrics JSONL.

Co-authored-by: Jason <jkbennitt@users.noreply.github.com>
@jkbennitt
jkbennitt marked this pull request as ready for review September 16, 2026 19:21
@jkbennitt
jkbennitt merged commit 046279a into main Sep 16, 2026
2 checks passed
@jkbennitt
jkbennitt deleted the cursor/typesafe-pareto-score-b44f branch September 16, 2026 19:24
@jkbennitt

Copy link
Copy Markdown
Member Author

Live metrics (post-merge smoke)

Ran against tip after CI green / merge path. TYPESAFE_API_KEY via env only — not in repo.

unsga3_pareto_score (this PR)

  • Model: jev-1.13.0
  • Mode: live HTTP POST /v1/systemone
  • Fixture: 8 synthetic ZDT1-like candidates (tools/typesafe-pareto/fixtures/zdt1_candidates.json)
  • Questions: 32 in one fan-out (3× Score + 1× Choice per candidate)
  • Latency: 529 ms wall
  • Usage: 4661 input / 771 output tokens
id f1 f2 feas constraint diversity exploit disposition composite
c0 0.00 1.00 Y 2.91 1.57 2.00 keep 0.911
c5 1.00 0.00 Y 2.83 1.64 1.99 keep 0.911
c4 0.75 0.13 Y 2.97 1.36 1.66 keep 0.850
c1 0.10 0.68 Y 2.97 1.17 1.63 keep 0.813
c2 0.25 0.50 Y 2.98 1.10 1.00 keep 0.739
c3 0.50 0.29 Y 2.93 1.06 0.65 keep 0.690
c6 0.50 0.60 Y 2.97 0.90 0.39 drop 0.642
c7 0.40 0.37 N 0.03 0.91 0.74 drop 0.238

Read: extremes kept; crowded mid-front dropped; infeasible c7 correctly dropped on constraint score.

Companion baseline (Petra stage-gate smoke, same day)

Separate @typesafe-ai/sdk / JS smoke on a SMARTS-only Challenge 0 row (not this repo):

  • 543 ms · 710 in / 114 out
  • evidence_class = computational_proxy (conf 1.0)
  • may_crown_stage_e noul = 0.06
  • slate_action = demote (0.88)

Logged for X/metrics artifact; NSGA objectives still authoritative here — Jev is additive only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants