[#531] Add configurable project scoring and judge deliberation - #532
Conversation
DVidal1205
added
Feature
|
Warning Review limit reachedNext included review available in 28 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: QUIET Plan: Team Run ID: 📒 Files selected for processing (20)
📝 WalkthroughWalkthroughAdds configurable judging with rubric validation, evaluations, score aggregation, visibility controls, revision history, private deliberation, and lifecycle state management. Adds database schema, API procedures, project filtering, judge tabs, officer command center views, audit coverage, import locking, and tests. Merge Risk: 🟡 Moderate · up to Judges can receive misleading feedback-sharing promises and stale deliberation views, while malformed evaluation data and invalid project–challenge relationships remain insufficiently constrained. These issues should be resolved before merge. 🚥 Pre-merge checks | ✅ 7 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (7 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 67 functions across 44 files. (6 skipped: 6 unsupported.) ✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (2)
apps/blade/src/app/_components/projects/judge-project-workspace.tsx-374-375 (1)
374-375: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winReplace the
(?)rating placeholder with readable text.Screen readers announce
(?)as punctuation. Use explicit text for the missing-rating state in both rating columns.As per path instructions for `apps/blade/**`: "Accessibility (alt text, ARIA, semantic HTML)".♻️ Proposed fix
const score = scoreByProject.get(project.id)?.scoped; return score?.value === null || score === undefined ? ( - "(?)" + <span className="text-muted-foreground"> + Not rated + </span> ) : (const score = scoreByProject.get( project.id, )?.overall; return score?.value === null || score === undefined - ? "(?)" + ? "Not rated" : `${score.value.toFixed(2)} (${score.count})`;Also applies to: 400-403
Source: Path instructions
apps/blade/src/app/judge/projects/loading.tsx-1-4 (1)
1-4: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winAdd a nested
Suspenseboundary with a static skeleton fallback aroundJudgeProjectsLoadingView; itsuseSearchParams()call is not covered by the route loading fallback.
🧹 Nitpick comments (2)
packages/api/src/tests/integration/judging-access.test.ts (1)
750-757: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winSensitive Data Exposure (CWE-359)
Reachability: Internal · Exploitability: Theoretical
Assert
isPublic: falsefor the private responses.This block checks only response values. It does not verify that the
"private"guest policy overrides the guest-providedisPublic: true.💚 Proposed addition
+ expect( + memberDetails.feedback.find( + (item) => item.value === "Always private response", + ), + ).toMatchObject({ isPublic: false }); + expect( + memberDetails.feedback.find( + (item) => item.value === "Private optional response", + ), + ).toMatchObject({ isPublic: false });Source: Path instructions
apps/blade/src/tests/projects/project-judge-privacy.test.tsx (1)
295-297: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAssert the completion state, not the Tailwind token.
ProjectBadgesusesevaluationCount > 0andtitle="1 evaluation"to expose completion. Replace the exact color assertion withexpect(screen.getByTitle("1 evaluation")).toBeInTheDocument().
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: QUIET
Plan: Team
Run ID: 8b422e8b-5968-4768-8b3a-ee8b4bea12f3
⛔ Files ignored due to path filters (1)
packages/api/src/tests/root/__snapshots__/api-surface.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (51)
.forge/features/judging-scores-and-deliberation/spec.md.forge/features/judging-scores-and-deliberation/srd.md.forge/features/judging-scores-and-deliberation/status.md.forge/features/judging-scores-and-deliberation/test-cases.mdapps/blade/src/app/_components/judging/evaluation-audit-panel.tsxapps/blade/src/app/_components/judging/evaluation-dialog.tsxapps/blade/src/app/_components/judging/judge-deliberation.tsxapps/blade/src/app/_components/judging/judge-submissions.tsxapps/blade/src/app/_components/judging/judging-configuration-panel.tsxapps/blade/src/app/_components/judging/judging-control-panel.tsxapps/blade/src/app/_components/judging/project-command-center.tsxapps/blade/src/app/_components/judging/project-score-dialog.tsxapps/blade/src/app/_components/projects/admin-project-workspace.tsxapps/blade/src/app/_components/projects/judge-project-workspace.tsxapps/blade/src/app/_components/projects/judge-projects-loading.tsxapps/blade/src/app/_components/projects/params.tsapps/blade/src/app/_components/projects/project-directory.tsxapps/blade/src/app/_components/projects/project-workspace-skeleton.tsxapps/blade/src/app/_components/shared/admin-navigation.tsapps/blade/src/app/admin/judging/loading.tsxapps/blade/src/app/admin/judging/page.tsxapps/blade/src/app/admin/projects/page.tsxapps/blade/src/app/judge/projects/loading.tsxapps/blade/src/app/judge/projects/page.tsxapps/blade/src/tests/admin/authenticated-shell.test.tsxapps/blade/src/tests/admin/hackathon-admin-navigation.test.tsxapps/blade/src/tests/projects/admin-projects-redirect.test.tsapps/blade/src/tests/projects/evaluation-audit-panel.test.tsxapps/blade/src/tests/projects/judge-deliberation.test.tsxapps/blade/src/tests/projects/project-judge-privacy.test.tsxpackages/api/src/projects-import.server.tspackages/api/src/routers/judging-scores.tspackages/api/src/routers/judging.tspackages/api/src/routers/projects.tspackages/api/src/tests/integration/judging-access.test.tspackages/api/src/utils/audit/coverage.tspackages/api/src/utils/judging/scoring.test.tspackages/api/src/utils/judging/scoring.tspackages/db/drizzle/0046_hot_zarda.sqlpackages/db/drizzle/meta/0046_snapshot.jsonpackages/db/drizzle/meta/_journal.jsonpackages/db/scripts/dev-db-backup-sanitizer.tspackages/db/src/schemas/knight-hacks.tspackages/db/src/tests/dev-db-backup-sanitizer.test.tspackages/db/src/tests/judging-schema.test.tspackages/db/src/tests/migration-lineage.test.tspackages/validators/src/audit.tspackages/validators/src/judging.tspackages/validators/src/projects.tspackages/validators/src/tests/audit.test.tspackages/validators/src/tests/judging.test.ts
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
|
Addressed every finding from the first review in 712c74e.
All four inline threads have replies and are resolved. Focused tests, @coderabbitai review |
|
🧠 Learnings used
|
|
@coderabbitai resolve |
✅ Action performedComments resolved and changes approved. |
1 participant
Why
Blade can import Devpost projects and grant room-based judging access, but the actual judging work still lives outside the product. Judges need a clear queue, an editable rubric submission, and a private way to prepare for deliberation. Officers need to configure that workflow without a yearly code change and audit what was submitted.
This slice turns the project table into the judging core while preserving the guest isolation introduced in #529. Sponsor guests stay locked to their room challenge. Authenticated judges keep their broader role-based access. Written feedback visibility now explicitly controls delivery to hackers, while authenticated judges and officers can review every response.
What
Closes: #531
Configurable rubric and lifecycle
Never share with hackers,Guest chooses hacker visibility, andAlways share with hackerspolicies. Authenticated judge feedback is always shared with hackers.Display all results. Guest score visibility never expands.Projects queue
Projects,Submissions, andDeliberationtabs to the judge workspace.Challenge ratingfor the active scope andRatingacross all scopes to authenticated members. Score sorting happens on the server and preserves pagination.See previously judgedis off by default and restores those rows.Submissions and feedback
Shared with hackersorNot shared with hackersfeedback.Private deliberation
Project command center
Command Centerwhile retaining the descriptive page heading./admin/projectscompatible through a server redirect into the command center Projects tab.Data and access controls
0046_hot_zarda.sql.Screenshots
No screenshot or Devpost CSV is committed to Forge. All review media below is hosted separately from the repository.
1. Authenticated judge project queue
Authenticated judges retain the normal Blade shell, room selection, unrestricted challenge filters, both score columns, and challenge completion badges. The screenshot enables
See previously judgedto show the completed AdaptLearn AI row beside the remaining queue.2. Authenticated judge rubric
The active challenge is called out at the top. Every quantitative question uses the configured 1 through 5 scale and touch-friendly controls.
3. Forced hacker feedback delivery
Authenticated judges do not receive a misleading privacy choice. The dialog states that the project team will receive the response.
4. Personal submissions
The Submissions tab gives each judge a single place to review and edit prior work.
5. Private deliberation workspace
Judges can build personal shortlists without changing any recorded score.
6. Searchable deliberation project picker
Only projects the current judge has evaluated are eligible, and the combobox supports text search.
7. Officer setup
The setup tab centralizes judging state, authenticated result visibility, and the data-driven rubric. This KH VIII fixture is already Open, so the rubric is visibly locked.
8. Protected project inventory
Ordinary imports stay available for unseen projects. Replacement and full deletion remain separate destructive actions after judging begins.
9. Rooms and live roster
The existing room model remains the source of guest challenge scope and now sits beside the rest of the judging controls. Officers can see account and QR guests together.
10. Evaluation audit history
Officers can inspect the current evaluation and every saved revision from one audit table.
11. Unskippable guest introduction
The projects shell stays blocked until the guest introduces themselves. The dialog explains that the name supports deliberation and identity verification and that responses are not shared from this introduction step.
12. Guest project queue
Guests see only the room challenge, its remaining projects,
Challenge rating, and the judged-project toggle. The Challenges and overall Rating columns are absent.13. Guest rubric context
The guest dialog explains that judges and officers can review every response and that the field-level policy controls what hackers receive.
14. Optional guest feedback
Optional guest feedback defaults to private. The toggle and status text name the recipient directly, so the effect is clear before submission.
15. Mobile authenticated queue
The server-rendered mobile shell keeps room and hackathon controls, all three workspace tabs, the judged-project toggle, and a card layout with usable touch targets.
Test Plan
Automated verification:
pnpm verify:precommitpnpm --filter @forge/db with-env drizzle-kit checkpnpm --filter @forge/blade buildgit diff --checkManual verification used KH VIII with 188 imported projects, 19 challenge records, and four judging rooms:
See previously judged.Checklist
pnpm db:generateand committed the generated files inpackages/db/drizzle/Deployment note: this slice adds no environment variable. It continues to use the
JUDGING_ACCESS_SECRETintroduced and provisioned with #529.