Skip to content

CLAUDE.md

Latest commit

 

History

History
309 lines (228 loc) · 76.9 KB

CLAUDE.md

File metadata and controls

309 lines (228 loc) · 76.9 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Commands

  • Install dependencies: pip install -r requirements.txt
  • Run the app: flask run (or gunicorn api.wsgi:app --log-file=- --log-level debug --preload --workers 1)
  • Run tests: pytest
  • Run a single test: pytest path/to/test_file.py::test_function_name
  • Run linting: pylint api/ common/ db/ model/ services/
  • Python environment: Miniconda with conda activate py39_ohack_backend (Python 3.9.13)

Architecture

A note about this codebase: This was originally taken from Auth0 and messages_views.py and messages_service.py were the original files we built from. Over time we have created a better structure for this backend service. Please do not update these two files anymore, you should consider the other code that exists first, and then add a new _views.py and a new _service.py if you're unable to find a good place to update the code as it relates to the file. If you find code in these two "messages" files that you plan to update, it's encouraged to migrate that code elsewhere to help iteratively move away from messages_views and messages_service.

Flask app with Blueprint-based API modules

The app is created via api/__init__.py:create_app(). Each API domain is a Blueprint registered there. The WSGI entrypoint is api/wsgi.py.

API module pattern

Each API domain lives in api/<domain>/ with a consistent structure:

  • <domain>_views.py — Flask Blueprint with route definitions. Routes use @cross_origin() and PropelAuth's @auth.require_user for authentication.
  • <domain>_service.py — Business logic called by views. Some older services live in services/ at the top level (hearts, users, volunteers, etc.).
  • tests/ — Tests for that domain.

API domains: messages, certificates, contact, github, hearts, judging, leaderboard, llm, newsletters, problemstatements, slack, store, teams, users, validate, volunteers.

Database layer (db/)

  • db/interface.py — Abstract DatabaseInterface base class.
  • db/firestore.py — Production implementation using Firebase Firestore.
  • db/mem.py — In-memory implementation (enabled via IN_MEMORY_DATABASE=True env var).
  • db/db.py — Singleton that selects the implementation and re-exports all DB operations as module-level functions. All service code imports from db.db.

Models (model/)

Data classes (User, Hackathon, Nonprofit, ProblemStatement, JudgeAssignment, JudgeScore, JudgePanel, etc.) with serialize()/deserialize() methods for Firestore document conversion.

Auth

PropelAuth via common/auth.py. Routes get the authenticated user from @auth.require_user and org context from the X-Org-Id header.

Caching

common/utils/redis_cache.py — Redis with TTLCache fallback. Used via @cached_with_key() decorator.

Logging

common/log.py — Structured JSON logging with get_logger(), info(), debug(), warning(), error(), exception() helpers. Supports colored console output.

Key environment variables

FLASK_APP=api, FLASK_RUN_PORT=6060, CLIENT_ORIGIN_URL, FIREBASE_CERT_CONFIG, OPENAI_API_KEY, PROPEL_AUTH_KEY, PROPEL_AUTH_URL, REDIS_URL (optional), IN_MEMORY_DATABASE (optional), ENVIRONMENT=test (for MockFirestore).

Profile data sources

  • Hackathon attendance is derived from the volunteers collection via services.volunteers_service.get_user_hackathon_attendance(user_id, email) — hackers always count; mentors/judges/volunteers require isSelected=True AND checkInTime. The legacy users.hackathons Firestore-ref array is no longer the source of truth.
  • Public profile (GET /api/users/<db_id>/profile/public) augments with hackathon_history (volunteer-derived), praises_count, and praises_recent (top 3) when the corresponding privacy field is "public".
  • Privacy fields (model.user.privacy_fields) include praises, which defaults to "public" (others default to True which is treated as private).
  • New public route: GET /api/users/<db_id>/praises?limit=&offset= returns paginated received praises (403 when private).

Deployment

Deployed to Fly.io (fly.toml, app: backend-ohack, region: sjc). Uses gunicorn (Procfile). Port 6060. ⚠️ As of June 2026 prod runs ONE sync gunicorn worker (Dockerfile CMD --workers 1, default sync class) — a single slow request blocks the entire API. Fix plan (workers/threads, caching, Sentry error sweep, traffic evidence): docs/perf-reliability-plan-2026-06.md.

Testing notes

  • Tests live in api/<domain>/tests/ or test/ at the repo root.
  • The app has heavy external dependencies (Firestore, OpenAI, PropelAuth, Slack, PIL). Tests must pre-mock these modules in sys.modules before importing service code.
  • ENVIRONMENT=test enables MockFirestore in the DB layer.

Code Style Guidelines

  • Python 3.9.13 (Flask backend)
  • Imports: Group standard library, then third-party, then local imports
  • Types: Use type hints for function parameters and return values
  • Naming: snake_case for variables/functions, PascalCase for classes
  • Error handling: try/except with specific exceptions
  • Linting: pylint (.pylintrc disables missing-module-docstring, missing-function-docstring, too-few-public-methods)

Gotchas (load-bearing — every one of these has bitten us)

common.utils.firebase needs load_dotenv() to have already run

common/utils/firebase.py reads FIREBASE_CERT_CONFIG at module import time (json.loads(safe_get_env_var(...))), with no lazy fallback — if that env var isn't in os.environ yet, the import raises JSONDecodeError immediately. common/utils/slack.py (and cdn.py, github.py, openai_api.py) call load_dotenv() at their own import time, which populates .env into the process env. Import common.utils.slack (or another dotenv-calling module) before common.utils.firebase in any new module that needs both — every existing service that imports both (api/mentors/mentors_service.py, api/submissions/submissions_service.py, api/peer_votes/peer_votes_service.py) does slack-then-firebase for exactly this reason. Getting the order backwards works fine in production (something else has already called load_dotenv() by the time your module loads) but breaks standalone script/test invocations that import your module directly.

Python 3.9 — no PEP 604 union syntax

Backend runs on Python 3.9. def foo() -> X | None: raises TypeError at import time, blowing up every endpoint that imports the module. Use Optional[X] from typing. Audit any new services/ module before committing.

PropelAuth user object — correct attribute names

auth_user (current_user from propelauth_flask) wraps the full PropelAuth User class. The attributes are NOT what they look like:

  • user.org_id_to_org_member_info — dict of {org_id: OrgMemberInfo}. NOT org_id_to_org_info (which silently returns None from getattr, leaving every admin check returning False).
  • Each OrgMemberInfo is an object, not a dict. Use .user_permissions (attribute) or .user_has_permission(perm) (method). org_info.get("user_permissions") always returns None.

Pattern for "is this user a global admin":

def is_admin(propel_user) -> bool:
    if not propel_user or not getattr(propel_user, "user_id", None):
        return False
    for org_info in (getattr(propel_user, "org_id_to_org_member_info", None) or {}).values():
        if org_info.user_has_permission("volunteer.admin"):
            return True
    return False

For most route protection, prefer the existing decorator: @auth.require_org_member_with_permission("volunteer.admin", req_to_org_id=getOrgId). Only roll your own check when combining multiple gates (per-resource editors list, etc.).

PropelAuth decorators do NOT inject user/org into the view

@auth.require_org_member_with_permission(...), @auth.require_user and @auth.optional_user only set the request-scoped auth_user proxy and then call the view with Flask's URL params alone. A view declared def admin_x(user, org, volunteer_id) raises TypeError: missing 2 required positional arguments: 'user' and 'org' on EVERY request — a 500, never a 403. Seven admin routes in api/volunteers/volunteers_views.py were written that way in Apr 2025 and stayed dead until Sep 2026 because their org gate also failed first (no req_to_org_id=getOrgId ⇒ required_org_id=None ⇒ 403 before the view ran); PR #280 fixed the gate on the select route and the TypeError surfaced in Sentry. Pattern: view params == URL params, read identity via auth_user.user_id inside the body. Every admin route needs req_to_org_id=getOrgId (the frontend sends X-Org-Id); the default resolver reads view_args["org_id"], which no route here has. Regression guard: api/volunteers/tests/test_volunteers_views.py dispatches through a real Flask app with common.auth stubbed (sys.modules swap — the real module calls PropelAuth at import) and asserts every view's signature matches its rule's arguments. Copy that fixture for other blueprints' route tests.

Firestore compound queries require composite indexes in production

.where("X", "==", v).order_by("Y") (where Y != X) needs a composite index declared in firestore.indexes.json AND deployed via firebase deploy --only firestore:indexes. The local Firestore emulator silently allows these queries; production Firestore returns 500 with "The query requires an index" — the route just hangs/errors.

Two options:

  1. Sort in Python after a single-field where (preferred when the result set is small): sorted([... for d in coll.where(...).stream()], key=lambda x: x["pos"]). No index needed because single-field equality is auto-indexed.
  2. Add the composite index to firestore.indexes.json AND deploy. Don't forget the deploy step — committing to the repo doesn't apply it.

set(..., merge=True) DEEP-merges map fields — removing a nested key needs DELETE_FIELD

ref.set({"some_map": {...}}, merge=True) does NOT replace some_map — it recursively merges, so a key you dropped from the Python dict stays in Firestore. To delete a nested key you must write firestore.DELETE_FIELD at that exact path: ref.set({"some_map": {key: firestore.DELETE_FIELD}}, merge=True). MockFirestore (test/ENVIRONMENT=test) REPLACES maps instead of deep-merging, so this passes locally and only breaks in prod. This caused the "mentor coverage cleared but stays checked" bug — toggle_mentor_coverage (api/mentors/mentors_service.py) popped the slug then wrote the dict, which never removed it. Pattern to copy is there now: write only the changed nested keys, DELETE_FIELD to remove. Mixing DELETE_FIELD sentinels and real values in the same nested map is allowed; DELETE_FIELD on a non-existent path is a no-op.

Lazy user profile creation in the users collection

A user authenticated via PropelAuth may NOT exist in the Firestore users collection. The collection is populated lazily — only when someone hits GET /api/users/profile or saves profile metadata. Never assume fetch_users() includes everyone with a propel_user_id referenced elsewhere (assignees, editors, mentions, etc.).

When resolving propel_id → display profile, fall back to services.users_service.get_oauth_user_from_propel_user_id(pid) for IDs not in the cached fetch_users() index. Cache the fallback aggressively (5 min minimum) — PropelAuth API calls aren't free.

OAuth provider response shapes (Slack vs Google)

get_oauth_user_from_propel_user_id(propel_id) returns the raw OAuth userinfo. Provider-specific fields:

  • Slack: https://slack.com/user_id (e.g. UC31XTRT5) is the Slack workspace user ID. https://slack.com/user_image_192 for avatar. email always present.
  • Google: no Slack ID. picture for avatar. email always present.
  • Detect by presence of https://slack.com/user_id — Google responses don't have it.

To send a Slack DM, pass the Slack user ID as channel: send_slack(message=..., channel="UC31XTRT5"). chat.postMessage opens (or reuses) a DM channel.

User.id vs User.user_id

  • User.id = Firestore document ID (used by /api/users/{id}/profile/public and the frontend /profile/{id} route).
  • User.user_id = PropelAuth user ID (the propel_id, what's stored in assignees[], editors[], etc.).

These are DIFFERENT VALUES. When bundling user data for the frontend, include both: {user_id: propel_id, db_id: firestore_doc_id, name, profile_image}. The frontend needs db_id to build profile links and user_id (propel) for matching against assignees/editors/mentions.

Volunteer applications (volunteers collection) — staff-owned fields

create_or_update_volunteer (services/volunteers_service.py) has exactly ONE production caller: handle_submit (api/volunteers/volunteers_views.py), backing only the self-service /api/{mentor,judge,hacker,volunteer,sponsor}/application/<event_id>/{submit,update} routes. It still persists the whole volunteer_data dict (no allowlist — new form fields flow through for free) except STAFF_OWNED_VOLUNTEER_FIELDS, which is stripped from the payload once, before the create/update branch. That covers both paths: on update set(merge=True) omits the keys so stored values survive; on create it stops a payload from overriding the isSelected: False seed to self-approve. The set covers approval (isSelected), check-in (isCheckedIn/checkedIn/checkInTime/checkInTimeList/checkOutTime/checkoutTimeList), refund bookkeeping (deposit_status, deposit_refund_*), certificates and sent_emails. Deliberately NOT in the set: stripe_payment_intent_id/deposit_amount_cents/deposit_disposition — the hacker Stripe Checkout return legitimately writes those via /update; adding them breaks deposits. The bug that motivated this (Aug 2026): all five frontend forms shipped isSelected: false from their initialFormData (sponsor hardcoded it), and the old guard only preserved the stored flag when the key was absent — so every application edit silently un-approved an approved mentor/judge/volunteer/sponsor. The same strip also closes a privilege-escalation hole that was reachable: before this, ANY logged-in user could POST isSelected: true to /submit and land an already-approved mentor or judge doc (the create path seeds isSelected: False then does volunteer_doc.update(volunteer_data)), granting MentorTeamPanel write access via user_is_mentor_for_event and survey-trust via get_user_event_roles. The submit/update routes were also @auth.optional_user and handle_submit had an elif 'user_id' in volunteer_data identity fallback — but the anonymous variant was NOT actually exploitable: send_slack_audit interpolates user.user_id before that fallback, and PropelAuth's LoggedOutUser has no user_id, so an unauthenticated POST raised AttributeError → caught → 400. That's an accident, not a control, so the routes are now @auth.require_user and handle_submit takes identity from the token ONLY — never a body user_id. Approval changes only via update_volunteer_selection (POST /api/admin/volunteer/<volunteer_id>/select) — the frontend's ONLY writer of isSelected (Sep 2026 hardening): volunteer.admin-gated like the refund routes, body selected must be a JSON bool (else 400), and the service busts hackathons_service.get_volunteer_by_event.cache_clear() (the in-process TTLCache behind GET /api/messages/admin/hackathon/<event>/<type> — redis pattern-clears don't reach it, so the admin roster showed a stale flag) plus a _notifications_disabled()-gated send_slack_audit. status is a separate field still written through the hackathon PATCH; never flip isSelected there. Regression tests: api/volunteers/tests/test_volunteers_service.py. Identity resolution is shared: find_volunteer_by_caller_identity(propel_user_id, event_id, volunteer_type) (services/volunteers_service.py) is THE 3-way resolver — propel UUID → PropelAuth email → OAuth user_id — used by handle_get (the GET route), create_or_update_volunteer (the write path), and api/mentors/mentors_service.py::_find_mentor_volunteer (delegates). Keeping read and write on the same resolver is load-bearing: when the write path matched propel UUID only, a user whose doc was stored under another identity shape saw their app on read, edited it, missed the write lookup, and fell into the CREATE branch — spawning a duplicate isSelected: False doc and orphaning the approved one. The email step uses the verified PropelAuth email only — never the form-payload email, which would let a caller hijack someone else's application by typing their address. Don't add a fourth copy of this lookup; delegate. (Surveys' get_user_event_roles is intentionally separate — it scans all volunteer_types in one pass.) Notification gate: _notifications_disabled() in volunteers_service.py suppresses the Slack/Resend fan-out (send_admin_notification_email, send_slack_volunteer_notification, send_volunteer_confirmation_email, send_mentor_checkin_notification) when ENVIRONMENT=test — before this, unit tests exercising create_or_update_volunteer posted REAL Slack messages and attempted REAL Resend sends. Mirror this gate on any new outbound-notification function in this service.

Confirmation-email calendar attachments (availability parser)

get_calendar_email_attachment_from_availability (services/volunteers_service.py) runs on every application submit/update to build .ics attachments from availability. It only understands the machine-generated slot format the mentor/volunteer forms emit ("Sunday, Oct 12: ☀️ Morning (9am - 12pm PST)"). The judge form's availability is a free-text field, so the function has an early guard: if the string contains no Weekday, Mon D prefix it logs INFO and returns []. Don't remove the guard or re-raise its logging to ERROR — that was the Aug 2026 Sentry noise ("CRITICAL: All patterns failed to match slot") firing on every judge application with availability text. Genuine structured-parse failures log a single WARNING per slot; the per-pattern cascade logs are DEBUG. Tests: api/volunteers/tests/test_volunteers_service.py (free-text skip + structured regression).

Hackathon requests (/hack/request → hackathon_requests collection)

create_hackathon / update_hackathon_request (services/hackathons_service.py) pass the full form payload to send_hackathon_request_email(..., request_data=json), which renders a "Your Submission" table in the confirmation email via _render_request_summary_html. Field labels/orders live in _REQUEST_SUMMARY_FIELDS (+ _RESPONSIBILITY_LABELS, _NONPROFIT_SOURCE_LABELS — the latter mirrors the frontend form's checkbox labels; keep in sync if HackathonRequestForm.js options change). All values are HTML-escaped (user input into email HTML); empty fields, donationPercentage: 0, and internal keys (status, created, agreements) are skipped. Tests: api/messages/tests/test_hackathon_requests.py.

Volunteer time tracking (/api/users/volunteering)

GET/POST in api/users/users_views.py → services/users_service.py. Both resolve identity through _resolve_and_ensure_user(propel_id), in this order so a broken OAuth provider token can NEVER block volunteering: (1) fetch_user_by_propel_id(propel_id) — direct Firestore lookup on the stored propel_id field, NO external call (covers everyone who has saved a profile); (2) the OAuth provider round-trip (get_oauth_user_from_propel_user_id → sub → fetch_user_by_user_id), the best source for the OAuth-format user_id + avatar, lazily creating a doc for new users; (3) the PropelAuth user-metadata fallback (_fetch_propel_metadata → auth.fetch_user_metadata_by_user_id) — RELIABLE, does NOT depend on the provider token — which resolves an existing doc by email (backfilling propel_id) or lazily creates one from the metadata (user_id set to the propel UUID since we lack the oauth-format id without the provider call; propel_id is the canonical match so step 1 hits forever after). The bug this fixes: the WRITE used to depend SOLELY on step 2; when get_oauth_user_from_propel_user_id returns None (expired/unavailable provider token, PropelAuth hiccup, or its 5-min negative cache) the write 404'd ("Couldn't log that time") while the read masked it by returning empty. Critical: get_profile_metadata (which creates the doc) ALSO depends on the OAuth round-trip, so a user whose OAuth has always failed may have NO doc at all — step 3 (metadata) is what resolves/creates them. fetch_user_by_propel_id/fetch_user_by_email live in db/{db,firestore,mem}.py (single-field equality queries — auto-indexed, no composite index). Logging: get_oauth_user_from_propel_user_id now logs the PropelAuth response BODY (truncated) on non-200 and a debug line when serving a cached miss — previously the root cause (e.g. "no linked OAuth connection", wrong PROPEL_AUTH_URL/KEY) was invisible during a tight retry window. Tests: api/users/tests/test_volunteer_resolve.py (6 cases). NOTE — date/locale is NOT a factor: <input type=date> always yields an ISO yyyy-MM-dd value regardless of the user's locale. get_volunteering_time now returns ([], 0, 0) (never None/404) so the page shows a clean zero-state, and filters in a SINGLE pass — an entry may carry commitmentHours, finalHours, or BOTH (manual logs send both), no concat/duplicate. save_volunteering_time accepts an optional timestamp (backdated manual logs) + manual:true flag; hours are float-cleaned, non-negative, capped at 1000.

Admin Email Templates (email_templates collection)

Powers the frontend /admin/communication template editor and the volunteer send-email dialogs. Blueprint: api/email_templates/email_templates_views.py (/api/admin/templates*, all volunteer.admin-gated); service: services/email_templates_service.py; seed data: services/email_templates_seed.py.

  • Layout: email_templates/{slug} main doc + email_templates/{slug}/versions/{000N} append-only content snapshots. version on the main doc = current version number; version doc ids are zero-padded for natural ordering.
  • Versioning rules: content-key edits (title/category/category_key/applicable_roles/message/icon) bump version + append a snapshot; status-only patches don't bump; revert never rewrites history — it copies the target version's content forward as a NEW version (change_note: "Reverted to version N").
  • Seeding: email_templates_seed.py holds the original 22 hardcoded frontend templates (GENERATED from frontend src/lib/messageTemplates.js — regenerate, don't hand-edit; editing it does not change live emails). Auto-seeds on first list call when the collection is empty; POST /api/admin/templates/seed re-inserts missing seed docs only — never overwrites admin edits.
  • DELETE is a hard delete (doc + versions). Deleted seed templates can be restored via /seed (history restarts at v1).

Hackathon roster import + audit scripts

Two scripts live in scripts/ for diagnosing and backfilling team rosters on /hack/<event_id>:

  • audit_hackathon_team_users.py --event-id <id> (read-only) — walks hackathons/{id}.teams[] -> teams/{id}.users[] and reports per-team member counts, dangling refs (team points to deleted user doc), and "ghost" users (no name + no propel_id = imported but never logged in).
  • import_hackathon_users_from_csv.py --csv <path> --event-id <id> --csv-type {registrants|projects|roster} [--apply] — dry-run by default. projects parses Devpost projects CSVs; the team-member triplet offset is resolved by header lookup (Team Member 1 First Name), since old 23-col exports have no "Team Number" column while newer 24-col ones do. Each parsed "email" is validated with the email regex — rows where the triplet shifted off-axis are skipped with a warning rather than written as bogus user docs. roster parses a generic team,email[,first_name,last_name,name] CSV for backfilling memberships; registrants just seeds user docs. Users are matched by email_address (case-insensitive). Imported users get imported=True, import_source, import_event_id, blank user_id/propel_id. Team membership writes are additive — never removes existing members. Re-runnable.
  • cleanup_bogus_imported_users.py [--event-id <id>] [--apply] — finds and removes the user docs left behind by the older off-by-one parse_projects bug. Fingerprint: imported=True AND propel_id="" AND email_address present but not a valid email AND import_source starts with projects-. For each matched user it prunes the doc-ref from every team's users[] that references it, then deletes the user doc. Dry-run by default. After running, re-run import_hackathon_users_from_csv.py --csv-type projects against the affected events to import the real members.
  • backfill_devpost_winners.py --event-id <id> --devpost-url <url> [--projects-csv <path>] [--apply] — scrapes the Devpost project gallery for EVERY project tile, flagging winners (aside.entry-badge img.winner). For each project it matches to a Firestore team via a layered strategy: teams.devpost_link exact-URL → team name (case-insensitive) → email-overlap via Devpost projects CSV (auto-discovered from /tmp/devpost_files/<event_id>/projects-*.csv). Two backfills happen in one pass: (1) any matched team with an empty devpost_link gets the gallery URL written; (2) matched WINNERS additionally get /software/<slug> fetched for prize text + member names, with prize strings mapped to status — "1st place" → FOUNDING_ENGINEERS, "Completion" or "2nd place" → COMPLETION_SUPPORT, anything else marked Winner → CATEGORY_WINNER (rank-based; multi-prize teams get the best status, all prize text retained in awards: []). Conflicts (team already has a different devpost_link) are logged but never overwritten. Unmatched winners exit with code 2 so a human notices; unmatched non-winners are listed for visibility but don't fail the run (typical for teams that registered only on Devpost). Only sets status, awards, winners_backfilled_at/source, and devpost_link; never touches users[]. Re-runnable. Adds beautifulsoup4 to requirements.

Resend API constraints

GET /emails (list) supports ONLY limit (max 100)/after/before — no recipient or date filter; results newest-first. Per-recipient status = Emails.get(id) using the resend_id stored in volunteer sent_emails, or webhooks. Rate limit is low single-digit req/s per team and shared with email sending.

Resend status architecture (implemented 2026-06-10)

  • Primary path: GET /api/admin/emails/resend-status — per-ID Emails.get with Redis cache (resend:status:{id}). Terminal events (delivered, bounced, complained, etc.) TTL 7 days; transient (sent, queued, etc.) TTL 120 s. Cap 100 IDs, 0.3 s throttle between calls.
  • List crawl: POST /api/admin/emails/resend-list — never blocks on page load. Always returns from cache immediately. Pass force: true to kick a background crawl. Date-bounded to 90 days; max 30 pages; 0.5 s between pages. Cache TTL 15 min fresh / 1 hour stale.
  • Admin UI: on volunteer load, bulk-fetches IDs from sent_emails via the status endpoint. "Sync from Resend" button sends force: true and shows a 30 s snackbar if the sync is still running.
  • Confirmation email tracking: send_volunteer_confirmation_email now accepts volunteer_id and appends a sent_emails record with recipient_type: 'application_confirmation' after sending.

Resend audience sync

scripts/sync_resend_audience.py --source {all|profiles|volunteers|mentors|judges|sponsors|helpers|leads} --audience "<name>" [--event-id <id>] [--selected-only] [--apply] — pulls emails from Firestore (users.email_address, volunteers.email filtered by volunteer_type, leads.email) and upserts contacts into a Resend audience (creates if missing). Dry-run by default. Re-runnable: lists existing audience contacts first and only POSTs new emails. Needs RESEND_API_KEY with audiences scope — the existing RESEND_WELCOME_EMAIL_KEY is send-only and will 401. Uses the deprecated resend.Audiences SDK class (now an alias for Segments) — fine for now, but if it breaks switch to resend.Segments. This logic is now ALSO ported into services/broadcasts_service.py (below) for the admin UI — keep loader/dedupe changes in sync or (better) treat the script as the ad-hoc CLI and the service as the source of truth.

Broadcasts blueprint (api/broadcasts/ + services/broadcasts_service.py, Aug 2026)

Powers the /admin/communication?tab=email Broadcast mode + the personalized bulk path. All routes volunteer.admin-gated, email_templates-style thin views. Routes under /api/admin/broadcasts: GET segments (list Resend segments), POST preview (dry-run per-source counts, no Resend writes), POST segments/sync (start background contact sync; 202 or 409 already_running), GET segments/<id>/sync-status (poll), GET|POST '' (list / create broadcast — draft by default, send:true/scheduled_at to send), GET <id>, POST <id>/send, POST batch-send (transactional Resend Batch, see below). Load-bearing details:

  • Sources spec consumed by preview+sync: {"sources":[{"type":"profiles"|"leads"|"volunteers"|"slack"|"contact_submissions", ...}], "custom_emails":[...]} — volunteers takes volunteer_type/event_id/selected_only; slack takes active_days (365 default / 10000 = everyone) and reuses get_active_users(days, admin=True) (deleted/bot/restricted already excluded there); contact_submissions takes inquiry_types (list, case-insensitive match on the doc's inquiryType; empty = all) + updates_opt_in_only (the form's receiveUpdates box). collect_contacts dedupes by lowercase email (first source wins the record; later sources fill blank names) and returns stats {per_source, custom_valid, custom_invalid, union_total, overlap_removed, contact_limit, over_limit}.
  • Segment sync is a daemon thread + redis (never inline — thousands of Contacts.create at ~20/s vs the 120s gunicorn timeout). Keys broadcasts:sync:{segment_id}:status (TTL 24h) / :lock (TTL 30min — self-heals worker death). Heartbeat every 25 contacts; get_sync_status reports state:"stalled" when a "running" status hasn't been touched for 120s (retry is safe — the sync diffs against _existing_segment_emails first, so it's idempotent). Contacts are CREATE-only — never update existing, never re-subscribe an unsubscribed contact.
  • API keys: segment/contact/broadcast ops use _resend_full_key() = RESEND_API_KEY with NO welcome-key fallback (it 401s). Batch send uses RESEND_WELCOME_EMAIL_KEY (Emails scope). resend.api_key is a module-global shared across threads — set it immediately before each call section.
  • From-address allowlist: RESEND_BROADCAST_FROM (default Opportunity Hack <updates@notify.ohack.dev>) + RESEND_BROADCAST_FROM_DOMAINS (default notify.ohack.dev,apply.ohack.dev). notifs.ohack.org is deliberately NOT allowlisted — its Resend domain verification is partially_failed (transactional sends still hardcode it; fix the DNS or migrate separately).
  • Contact-cap guardrail: RESEND_MARKETING_CONTACT_LIMIT (default 1000 = the free marketing tier OHack is on as of Aug 2026; Resend bills marketing by CONTACT COUNT, not sends — 5k=$40/mo, 10k=$80/mo). Preview/sync report over_limit; sync proceeds and captures per-contact failures. Broadcast HTML gets a {{{RESEND_UNSUBSCRIBE_URL}}} footer appended server-side if missing (Resend rejects broadcasts without it).
  • POST batch-send ({subject, recipient_type?, recipients:[{email,name,message}]}, ≤500/request): renders each pre-personalized message through the same HTML shell as _send_email_to_user and sends via resend.Batch.send in chunks of 100 (transactional quota — this replaced the frontend's one-request-per-recipient loop for email-only recipients). [QRCode:...] messages are rejected per-recipient (Batch has no attachments); registered-user sends stay on /api/admin/<id>/message (Slack DM side effect). Falls back to sequential Emails.send when the SDK predates Batch (local env note: requirements pins resend 2.22.0 but the conda env had 2.3.0 — pip install -U resend==2.22.0).
  • Contact management (quota reclaim): GET /admin/broadcasts/contacts (full account-level crawl, 60s redis cache broadcasts:contacts:index, ?force=true; returns contacts/total/unsubscribed_count/contact_limit/over_limit), POST /admin/broadcasts/contacts/prune (modes unsubscribed|emails|all; ONE global background job, lock broadcasts:contacts:prune:lock + status ...:prune:status, same stall/heartbeat semantics as sync), GET .../prune-status. Deletes use resend.Contacts.remove(email=...) with NO audience_id → removes the GLOBAL contact, which is what frees marketing quota (unsubscribed contacts still count against it). Sync + prune both bust the contacts cache on completion.
  • All sends + sync completions are gated by a local _notifications_disabled() (ENVIRONMENT=test → simulated:true) and audited via send_slack_audit. Tests: api/broadcasts/tests/test_broadcasts_service.py.
  • userlist() in common/utils/slack.py is now redis-cached (slack:userlist, TTL 600s, decorator ABOVE the RateLimiter so cache hits skip the blocking limiter) — one crawl serves every active_days filter; clear_slack_cache() clears the new prefix too. All userlist consumers now see up-to-10-min-stale member data (fine — the "activity" field only changes on profile updates).

The frontend /hack/<event_id> page's "Team Members:" list is teams.users[] (DocumentReferences). The bug pattern that motivated this: a team's users[] only contains the user who created the team on ohack.dev; everyone else registered via Devpost/JotForm and was never linked. Use audit first to confirm, then import ... --csv-type roster (or projects for old Devpost exports) to backfill.

Event Feedback Surveys (surveys collection)

Post-event / live-event feedback, stored in a NEW surveys collection — deliberately distinct from the peer-to-peer feedback collection. Blueprint api/surveys/surveys_views.py + services in api/surveys/surveys_service.py. Routes (/api/surveys/<event_id>/...):

  • GET context — public (@auth.optional_user). Returns mode (live|post|upcoming), the caller's eligible roles, primary_role, requires_captcha, already_submitted, and a light event block.
  • POST responses — public (@auth.optional_user). Logged-in volunteers who are eligible for the event are "trusted" and skip CAPTCHA; everyone else (nonprofit partners — no flag yet — and anonymous) must pass reCAPTCHA via the shared verify_recaptcha from api.contact.contact_service.
  • GET responses / GET summary (per-event) and GET /api/surveys/overview (cross-event) — volunteer.admin-gated. overview is one scan of the surveys collection grouped by event_id (get_cross_event_survey_overview): per-event count/by_mode/by_role, averages of the two universal scales (overall_rating, would_return), first/last_response, joined with hackathon title/dates/tz (lazy-import get_hackathon_list). Aggregates only — no per-response data, no PII. Powers the "Compare events" sub-view; static /surveys/overview doesn't collide with /surveys/<event_id>/....

compute_event_mode is timezone-aware off start_date/end_date (mirrors the frontend isHackathonExpired; missing dates → live). Eligible roles come from the volunteers collection via get_user_event_roles — hacker counts on application, mentor/judge/volunteer/sponsor require isSelected — matched 3 ways (propel UUID / email / OAuth user_id) like handle_get. Role scope: a trusted volunteer may submit ONLY for a role they're selected for; everyone else is restricted to nonprofit (allowed_roles_for). Enforced in submit_survey_response (403 on mismatch) and surfaced as allowed_roles in the context response so the frontend can scope its selector. Logged-in responses upsert by doc id {event_id}__{mode}__{propel_user_id} with a full set() (NOT merge=True, which would deep-merge the answers map and keep cleared keys) carrying created_at forward; anonymous responses are random-uuid docs. Heavy/external deps (get_hackathon_by_event_id, verify_recaptcha, slack) are lazy-imported inside functions so the module imports cheaply (testable; pure date/mode logic covered in api/surveys/tests/). Question IDs/catalog live frontend-side; backend stores answers verbatim. Live-mode submissions ping the #feedback Slack channel; post-mode is audit-only.

Admin feedback review (api/feedback/)

Read-only admin aggregation for the frontend /admin/feedback dashboard. New blueprint api/feedback/feedback_views.py (NOT messages_views — frozen) + api/feedback/feedback_service.py (the admin READ side; writes still live in services/feedback_service.py + services/onboarding_service.py + the surveys domain). Both routes volunteer.admin-gated, ?limit= (default 500, cap 2000):

  • GET /api/admin/feedback/peer — peer-to-peer feedback collection, newest first, giver/receiver names resolved via a one-shot fetch_users() directory (giver hidden when is_anonymous); light by_relationship/by_role summary.
  • GET /api/admin/feedback/onboarding — onboarding_feedbacks collection, newest first, + rating & ease distributions. Keeps clientInfo.userAgent, drops the IP. Field shape (load-bearing for the admin UI): maps contactForFollowup → contact: {willing, firstName, email} (the form stores firstName+willing, NOT name — don't revert to name); overallRating 0 = unrated (skipped in the average); timestamps are normalized to ISO, stripping a __Timestamp__ export sentinel (see scripts/sync_hackathons_from_csv.py) if present. Event surveys reuse the api/surveys admin routes (/responses, /summary, /overview) — not duplicated here.

Praise-bot remote config (api/praisebot/, praise_bot_config collection)

Config store for the Slack praise-bot (repo ohack-slack-bot/praise-bot): the bot polls it every ~60s so channels/repos/crons/toggles are managed from /admin/praise-bot instead of Fly.io env vars. Blueprint api/praisebot/praisebot_views.py + praisebot_service.py.

  • Doc types in one collection: fixed doc id global (dry_run/llm_enabled/timezone), github_watcher (source mode: hackathon|repos, digest + optional rollup crons), calendar_reminder, and singleton community (intro matchmaker + weekly digest). Audit fields created_at/updated_at/updated_by on every doc.
  • GET /api/praise-bot/config — bot-facing, authed via X-Api-Key against BACKEND_BOT_CONFIG_TOKEN (falls back to BACKEND_PRAISE_TOKEN) through the shared common/utils/api_key.py:check_api_key (hmac.compare_digest; new code should use this instead of the inline checks in messages_views). Returns configured: false when the collection is empty → bot uses its env defaults.
  • GET/POST /api/praise-bot/admin/config, PATCH/DELETE /api/praise-bot/admin/config/<doc_id> — volunteer.admin-gated. Validation is whitelist-per-type (_ALLOWED_KEYS — the enforcement point that keeps secrets out of docs); crons validated as 5 fields (bot re-validates with cron.validate()); repos normalized to owner/repo; global is upsert-only (no DELETE); community is a singleton (POST 400s if one exists). source.orgs is accepted/stored but ignored by the bot until org-watching ships. 15s TTL cache on the assembled config, cleared on mutation. Tests: api/praisebot/tests/ (mockfirestore, run with ENVIRONMENT=test).

Volunteer job board (api/jobs/, job_listings + job_applications collections, Aug 2026)

Powers the frontend's /jobs pages and /admin/jobs. Blueprint api/jobs/jobs_views.py + jobs_service.py.

  • job_listings doc id = slug (immutable after create; POST 409s duplicates). Statuses draft|published|hidden|closed — public list returns published+closed (lean fields), single-get 404s draft/hidden but returns closed (shared links render a closed panel). posted_at auto-stamped on first publish. 300s TTL caches (get_public_listings/get_public_listing) cleared on every admin write. Validators + ALLOWED_JOB_*/JOB_LISTING_ADMIN_KEYS constants live in common/utils/validators.py.
  • POST /api/jobs/<slug>/apply is @auth.require_user + @RateLimiter + recaptcha (imports volunteers_service verify_recaptcha, keeps the FLASK_ENV=development bypass): validates via validate_job_application (visa_ack must be True, work sample ≥ 200 chars — keep in sync with the frontend's MIN_WORK_SAMPLE_CHARS), verifies resume_url is under the caller's own job_applications/<db_id>/ CDN prefix and video_url is own-CDN (users/<db_id>/, the bio-video mint) or an ALLOWED_VIDEO_LINK_HOSTS link, then 409s if the user already applied to that listing. Resume mint POST /api/jobs/apply/resume-upload-url reuses common/utils/cdn.generate_signed_upload_url (PDF only, 10MB, resolves the user via users_service _resolve_and_ensure_user).
  • Emails (Resend, all behind the local _notifications_disabled() mirror): applicant confirmation with the reply-within-5-days responsiveness ask (reply_to: questions@ohack.org), FYI to questions@ohack.org, and warm accept/reject decision emails via POST /api/jobs/admin/applications/<id>/decision ({decision, personal_note?}; records into sent_emails ArrayUnion + status_history). Admin routes are volunteer.admin-gated; application PATCH allowlist is status/admin_notes only.
  • Seed: scripts/seed_job_listings.py (dry-run default, --apply writes the three Fall 2026 roles as drafts, skips existing slugs so admin edits survive re-runs; validates against validate_job_listing so seed/validator drift fails loudly).

Public portfolio (profile → portfolio, Aug 2026)

The public profile payload (GET /api/users/<id_or_slug>/profile/public) is now the "portfolio" payload. Load-bearing contracts:

  • model/user.py is the control surface. metadata_list += bio, headline, portfolio_links (savable via the generic profile POSTs — BOTH api/users and the legacy api/messages paths, which each call _sanitize_portfolio_metadata from users_service: bio ≤2000, headline ≤80, links ≤10 × {label ≤40, url} with https autoprefix + whitespace-URL rejection since validate_url lets spaces through). privacy_fields += bio, bio_video_url, portfolio_links, teams, certificates, github_history, hearts — all default private; get_privacy_settings() backfills lazily. safe_public_fields += id, profile_slug (they ARE the public URLs). headline rides the bio privacy toggle. profile_visibility (private|public, DEFAULT_PROFILE_VISIBILITY="private") is always emitted; private does NOT strip content — it only tells the frontend to noindex (old links keep today's per-field rendering; public = indexable + sitemap and requires a claimed slug, enforced in set_profile_visibility).
  • Profile fields have ONE source of truth: PROFILE_FIELD_SPECS in model/user.py (Aug 2026 profile-stack retirement). Adding a flat profile field = one spec entry (name, default, owner_editable, persisted) (+ a privacy_fields entry if privacy-gated). The registry generates metadata_list, OWNER_EDITABLE_FIELDS (what POST /profile accepts), PROFILE_PERSISTED_FIELDS (what the generic upsert writes), and serialize_profile_fields(); users_service.build_profile_response(user) is THE canonical own-profile response (hackathons attendance-derived, NOT the deprecated users.hackathons refs). Guards in api/users/tests/: test_field_registry.py fails CI when any hand-list (privacy/pii/safe, _ADMIN_PROFILE_LEAN_FIELDS) drifts; test_profile_roundtrip.py asserts every editable field survives POST→GET. volunteering is deliberately NOT in the generic write set (dedicated update_user_volunteering — a profile save racing a volunteering log must not clobber it); propel_id/bio_video_url/profile_slug/profile_visibility are never owner-editable via metadata. Profile get/save resolve identity via _resolve_and_ensure_user (3-tier; OAuth outage can't 404 the profile). The legacy /api/messages/profile* routes are THIN DELEGATES onto this stack (parity-tested in api/messages/tests/test_profile_delegates.py) pending final deletion once the migrated frontend deploy is confirmed live — the old hand-built _old bodies are gone; don't add fields to delegates.
  • Dedicated writers only for profile_slug (POST /api/users/profile/slug, check at GET .../slug/check/<slug>), profile_visibility (PATCH /api/users/profile/visibility), bio_video_url (POST /api/users/profile/bio-video) — never add these to metadata_list (uniqueness/validation would be bypassed via merge writes).
  • Slugs (services/user_slug_service.py): user_slugs/{slug} pointer collection, slug IS the doc id → uniqueness via DocumentReference.create() (MockFirestore fallback: get+set). Old slugs stay as is_primary: False aliases forever (links never break, no slug-jacking); rename throttled 1/24h via pointer created_at, max 5 pointers/user; RESERVED_SLUGS + regex ^[a-z0-9](?:[a-z0-9-]{1,28}[a-z0-9])?$ + reject 32-hex (db-id shadowing). Resolution: _get_user_profile_by_db_id_or_slug tries the doc id first (db-id always wins), then the pointer — wired into profile/public, privacy-settings, and praises getters.
  • Attach functions in users_service (same try/except pattern as _attach_hackathon_history): _attach_teams (raw-doc team refs → db.get_all — deserialize drops teams; allowlisted team fields + trimmed event via get_hackathon_by_event_id; @redis_cached("portfolio:teams", 900)), _attach_certificates (heart certs from history.certificates + git-fame via get_certificates_by_github_username; never expose author_email), _attach_github_contributions, _attach_hearts (hearts_service.get_hearts_summary = what+how sum only, matches frontend HeartGauge; tiers mirror src/lib/heartTiers.js — keep in lockstep).
  • Caching: whole payload @redis_cached("portfolio:profile", 300) (get_portfolio_profile); clear_portfolio_caches() (pattern-clears portfolio:* prefixes) is called from profile save (both paths), privacy PATCH, visibility PATCH, slug claim, bio-video set, and give_hearts_to_user. redis_cached never caches None and silently skips non-JSON-serializable results.
  • Certificates by user: cert docs now stamped with github_username at generation (_extract_github_username: noreply-email regex, fallback bare author_name, lowercased). GET /api/certificates?github=<username> (bare prefix route; sort/dedupe in Python — no order_by, avoids a composite index). Run scripts/backfill_certificate_github_usernames.py --apply once so pre-existing certs match.
  • GitHub contributions: get_github_contributions_for_user no longer hardcodes the 2025 org — collection-group query on login only. Requires the github_contributors.login COLLECTION_GROUP fieldOverride in firestore.indexes.json — deploy with firebase deploy --only firestore:indexes BEFORE shipping. Timestamps ISO-ified for redis; get_github_profile TTL 10s→1h.
  • Bio video: never proxy bytes through Flask. POST /api/users/profile/bio-video/upload-url mints a V4 signed GCS PUT URL (common/utils/cdn.py::generate_signed_upload_url, x-goog-content-length-range enforces the 100MB cap server-side; client must echo required_headers). The setter accepts own-CDN users/{db_id}/… URLs (blob verified via get_blob_metadata) or YouTube/Vimeo/Loom hosts; replaces best-effort-delete the previous own-CDN blob. One-time ops: set bucket CORS to allow PUT from www.ohack.dev/ohack.dev/localhost (gcloud storage buckets update gs://$GCLOUD_CDN_BUCKET --cors-file=…).
  • Sitemap feed: GET /api/users/portfolio/sitemap (public, @redis_cached("portfolio:sitemap", 3600)) → [{slug, last_login}] for profile_visibility == "public" users; consumed by the frontend server-sitemap.xml.js.
  • Security fixes shipped with this: get_profile_by_db_id now returns only safe_public_fields (was leaking propel_id + privacy-ignoring github); get_praises_about_user privacy-gates the raw-Slack-id praise route inside the service (messages_views is frozen); POST /api/certificates/generate requires login, and its slack_channel batch mode requires volunteer.admin (checked manually via org_id_to_org_member_info — pattern from hackathon_planning_service.is_admin).
  • Tests: api/users/tests/test_portfolio.py (slug validation/claims/throttle, visibility gating, privacy matrix, sanitizer) + test/services/test_portfolio_helpers.py (hearts summary/tiers, cert username extraction). Run per-directory (pytest api/users/tests test/…) — running pytest api/ wholesale hits a pre-existing tests-package name collision.

Problem statement "Who's helping" roster (Sep 2026, frontend #359)

GET /api/problem-statements/<id>/helpers (api/problemstatements/problem_statement_views.py, public) → services/problem_statements_service.py::get_problem_statement_helpers. The raw helping array is append-only history ({user: <db id>, slack_user, type, timestamp}) and real docs carry the same person 2–3× from double clicks, so normalize_helping_entries collapses to one row per person (earliest timestamp → since, latest type wins, junk entries dropped, oldest first) and _enrich_helpers_batch attaches name/nickname/profile_image with ONE db.get_all (same public-safe field set as team rosters — no email/propel_id). 60s TTL cache (_helpers_cache, clear_helpers_cache(ps_id)) cleared inside save_helping_status, which now also updates a returning helper's entry in place (keeps their original timestamp, collapses their duplicates) instead of appending. 404 when the doc doesn't exist. Tests: api/problemstatements/tests/test_helpers.py (run per-directory). The frontend keeps a counts-only fallback when this route is missing, so deploy order doesn't matter.

Project submissions + Hackers' Choice (Sep 2026)

Backend half of replacing DevPost with an in-house team dashboard (frontend plan: docs/plans/team-dashboard-devpost-replacement.md). Judging itself (rubric, rounds, scoring, results) is completely unchanged — the only judging-facing addition is the team's demo video (see the judging bug-fix note below). Two new blueprints, both registered in api/__init__.py after broadcasts_views:

api/submissions/ — team project write-ups + submission deadlines

Self-serve, deadline-aware writes to a team's project_* fields, split deliberately from api.teams.teams_service.edit_team (the admin write path — no deadline gate, no membership check, org-permission gated at the route; an admin can override any project_* field, including project_submission_status — validated against draft|submitted|late, 400 on anything else, no write — via PATCH /api/team/edit; project_tagline/project_story get the same sanitize_markdown treatment there as the self-serve path, and a status change stamps project_updated_at). Every write goes through _authorize_team_write(propel_user_id, team_id, admin, enforce_deadline): 404 unknown team → 403 not_team_member (non-member, non-admin) → 409 submissions_closed{deadline,late_until,now} once the event's submission window has closed, unless admin=True or the caller opted out (enforce_deadline=False, used only by mentor-availability). Full contract, sanitization rationale, and CDN-image validation rules: api/submissions/README.md.

submit_project checks the already-submitted idempotent path before the deadline gate — a team that submitted on time and revisits the dashboard after the window closes still gets 200 already_submitted:true, never a spurious 409; only a not-yet-submitted team is blocked past close (unless admin). self_serve_team_edit (bridge for /devpost and /demo-video) calls this module's own clear_cache() after delegating to edit_team — edit_team alone only busts the generic per-function caches, not services.hackathons_service's separately-cached get_single_hackathon_event (10-min TTL), so without this the event page kept showing a stale DevPost link/demo video for up to 10 minutes after a self-serve save.

New team-doc fields (absent ⇒ legacy team, no dashboard implied): project_tagline, project_story (raw markdown; the frontend renders it via react-markdown without rehype-raw, so sanitize_markdown in common/utils/validators.py is defence-in-depth, not the primary XSS boundary — it loops its tag-strip to a fixpoint so a nested bypass like <scr<script>ipt> can't reassemble into a live tag on a single pass, matches on*= attributes glued to a / as well as whitespace [<img/onerror=...>], neutralizes javascript:/vbscript:/data: targets whether quoted or unquoted, also neutralizes the same targets in markdown link/image syntax [](javascript:...) → ](#)], and deliberately preserves generic < so List<String>/Map<K,V> in a write-up survives), project_built_with, project_links, project_thumbnail_url, project_images, project_updated_at, project_submitted_at, project_submission_status (draft|submitted|late), mentor_help_wanted (absent ⇒ True, a signal-only "open to mentors / heads-down" toggle with no deadline gate and no other behavior change).

New hackathon-doc field: deadlines ({submission, late_submission_until, voting_opens, voting_closes}, each a tz-normalized ISO string or None). common/utils/validators.py::validate_deadlines normalizes naive datetimes into the event's own timezone (normalize_deadline_iso) and enforces submission <= late_submission_until / voting_opens < voting_closes when both sides of a pair are present in the same payload; an unknown key or a bad value skips the whole deadlines field in validate_hackathon_data_partial (goes into skipped_fields) rather than partially applying it. services/hackathons_service.py::save_hackathon turns an explicit None into a Firestore DELETE_FIELD sentinel on an update (so a cleared deadline actually clears, not just leaves the old value under merge=True) but simply omits it on create (nothing to delete yet). deadlines: {} (and top-level deadlines: null) on an update is a no-op, not a clear-all — an empty map merges zero sub-fields into the stored deadlines map under merge=True, leaving it untouched; to clear one deadline, send {key: null} for that key specifically. get_single_hackathon_event strips project_story off every team in the response (it can run to ~20k chars and the endpoint already fans out to every team on the event) — the dashboard/team page fetch it via the per-team routes instead. Both compute_submission_window (here) and compute_voting_window (api/peer_votes/) re-parse the stored deadline strings through normalize_deadline_iso before comparing them against now — a naive or "Z"-suffixed stored value used to raise (TypeError comparing naive-vs-aware, or ValueError on Python 3.9/3.10's fromisoformat rejecting "Z") and reach the caller as a 500; an unparseable value is now logged and treated as absent (no_deadline / closed) instead.

Deadline reminders (build_reminder_message, send_deadline_reminders, send_due_reminders_for_current_events, same file): a per-team Slack nudge naming only what THAT team still owes (never nags an already-submitted team — returns None). Idempotency key is reminders_sent[f"{kind}_{hours_before}h"] on the hackathon doc; only_if_due=True (used by the hourly cron, .github/workflows/deadline-reminders.yml, X-Api-Key: BACKEND_CRON_TOKEN via common/utils/api_key.check_api_key) no-ops outside a one-hour-wide [deadline - hours_before, deadline - hours_before + 1h) window instead of erroring, so the cron can safely call POST /api/hackathons/deadlines/remind-due every hour for every current event × {24,6,1}h without spamming teams the other 23 hours. (The window used to extend all the way to the deadline itself — [deadline - hours_before, deadline) — which meant a deadline set with only a few hours' notice fell inside BOTH the 24h and 6h due windows on the very first cron tick and fired both reminders at once; each tier now gets its own narrow hour-wide slot, and a deadline set too close to fire a given tier's slot simply skips that tier rather than double-firing.) The admin "Send reminder now" button hits POST /api/hackathons/<event_id>/deadlines/remind with a Bearer token (@auth.optional_user + is_admin(auth_user)) instead of the API key. BACKEND_CRON_TOKEN must be set on both Fly (backend env) and as a GitHub Actions secret — this PR does not set either; do that before merging or the cron 403s.

api/peer_votes/ — Hackers' Choice (assigned-slate approval vote)

Anti-popularity peer award, deliberately NOT a raw "vote for your favorite": each eligible voter (an isSelected hacker; constraints.peer_vote_requires_submission additionally requires the voter's own team to have submitted) gets a deterministic, exposure-balanced slate of constraints.peer_vote_slate_size (default 5) submitted projects, never their own team — seeded on sha256(f"{event_id}:{propel_id}") so the same voter always sees the same slate, sorted by current exposure ascending so under-shown projects surface first. Voters approve, not rank: pick up to constraints.peer_vote_max_picks (default 2, clamped in _settings() to at most slate_size - 1 at READ time — validate_hackathon_data_partial only enforces that relationship when both fields are present in the same PATCH payload, so a doc can end up with an inconsistent stored pair; every voter-facing route re-clamps rather than trusting the stored value). Scoring is the Wilson score interval lower bound (z=1.96) of approvals/shown, not a raw rate — wilson_lower_bound(0,0)==0, (5,5)≈0.566, (1,1)≈0.207 — so a team shown to 2 people who both approved doesn't outrank a team shown to 40 with a 90% rate. shown in compute_results counts cast ballots (non-voided, with picks) that included the team in their slate — NOT the raw exposure-doc count, which only reflects how many slates the team was ever persisted into (someone who opened the page but never voted). The raw count is kept separately as exposure_shown. No tallies are ever shown to a voter, only to admins (GET .../peer-vote/results, which also returns a ballots_detail: [{voter_propel_id, voted_at, voided, picks_count}] list — no names, emails, or picks — so the admin UI can drive void without seeing who voted for what). A voided ballot's own slate page renders status: "voided" with picks: null — never "voted" with stale picks. One ballot per voter (deterministic doc id f"{event_id}__{propel_id}" in the top-level peer_votes collection, / replaced with _), re-votable until close (both the pick-update in submit_ballot and the void in void_ballot do a full set(), not set(merge=True) — the whole doc is rebuilt from the existing one so it stays a complete, self-describing record), admin-voidable, admin-publishable. POST .../peer-vote/publish appends "Hackers' Choice" to the winning team's awards[] once (idempotent) and writes a public summary doc at hackathons/{doc}/peer_vote/summary (exposure counts live at hackathons/{doc}/peer_vote/exposure) — but refuses with 409 no_ballots when results["ballots"] == 0 OR the computed rank-1 team has zero approvals (compute_results emits a row for every submitted team regardless of vote count, so with no ballots cast every row ties at a Wilson score of 0 and the sort falls through to team name, which would otherwise crown an arbitrary "winner" of a vote nobody voted in). constraints.peer_vote_enabled (default False) is read on every voter-facing route — a disabled or unconfigured event returns {"status": "disabled"} from the slate route and 403 peer_vote_disabled from the ballot route, regardless of anything else. Full contract: api/peer_votes/peer_votes_service.py module docstring + function docstrings.

Firestore transaction note: get_slate's first-ever materialization for a voter (persisting the slate + incrementing exposure) runs inside _in_transaction(db, body), which re-checks the ballot doc's existence inside the transaction (not just the optimistic outer read before it) so two near-simultaneous requests from the same voter can't double-build a slate or double-increment exposure. _in_transaction is the one seam tests monkeypatch (real @firestore.transactional needs a live Firestore client) — swap it for lambda db, body: body(FakeTransaction(...)).

Judging — video-only change + 3 bug fixes (api/judging/judging_service.py)

No rubric, scoring, round, or results logic changed. get_team_details/format_team_for_judge now also return demo_video_url (the real field a team's dashboard writes) and fill the legacy video_url key from it (team.get('demo_video_url') or team.get('video_url', '')) so both old and new judge-page code read a working value. Bugs found and fixed along the way (all pre-existing, none related to the video change):

  • get_bulk_judge_details always returned an empty judges list — it called an undefined name, fetch_judge_scores_by_event (the correctly-named fetch_judge_scores_by_event_id was imported but never used), which NameError'd straight into the function's own blanket except. One-line rename fixes it.
  • update_judge_assignment_details (PUT /api/judge/assignments/<id>) always 400'd "Assignment not found" — it looked assignments up via fetch_judge_assignments_by_judge_id("") (an empty judge_id can never match a real assignment) instead of by the assignment's own id. Added fetch_judge_assignment_by_id (direct doc-get, db/firestore.py + db/db.py) and used that instead.
  • api/github/github_views.py's /issues route let a request through with no org (the service then 400'd with a 200 status, since the view unconditionally returned jsonify(...) with no status code), and logged len(issues) where issues is the service's response dict — that logged the dict's key count, not the issue count. Both fixed while adding /activity (see below).

GET /api/github/activity?org&repo — team dashboard "Code activity" card

common/utils/github.py::get_repo_activity(org, repo) makes exactly 3 GitHub API calls (rate-limit budget, not completeness — this is a live-ish snapshot): g.get_repo(f"{org}/{repo}"), repo.get_commits().get_page(0) (first page only, ≤100 commits; a 409 "empty repository" is a valid all-zeros result for a fresh team repo, not an error), repo.get_pulls(state="open").totalCount. Contributors are derived from that single commit page (a Counter over each commit's author), not a separate stats call. api/github/github_service.py::get_github_activity validates org/repo against ^[A-Za-z0-9_.-]{1,100}$, caches successes only for 300s (_ACTIVITY_CACHE, separate from the existing _ISSUES_CACHE), and translates UnknownObjectException→404 repo_not_found, RateLimitExceededException→503 github_rate_limited, anything else→502 github_unavailable. Commits by activity_excluded_logins() (env GITHUB_ACTIVITY_EXCLUDED_LOGINS, default gregv — the GITHUB_TOKEN owner who seeds LICENSE + README via create_github_repo) are dropped before any counting, matched on GitHub login or, for unlinked commits, the git author name; otherwise every new team repo showed 2 commits and the dashboard's "Push code" row was done before the team pushed anything.

GET /api/volunteer/<event_id>/me?type=hacker

Widened the existing mentor self-check route. services/volunteers_service.py::get_volunteer_self_status(propel_user_id, event_id, volunteer_type) mirrors api.mentors.mentors_service.get_mentor_self_status's shape/leanness ({f"is_{type}": bool, "volunteer": {"name","isSelected"}|None}) for any volunteer_type, via the existing find_volunteer_by_caller_identity resolver. Backs the team dashboard's "am I an approved hacker for this event" gate and Hackers' Choice eligibility. type=mentor still delegates to the original mentor-specific service unchanged.

Security fix: /devpost and /demo-video now check team membership

POST /api/team/<id>/devpost and POST /api/team/<id>/demo-video (api/teams/teams_views.py) used to call edit_team directly with no membership check at all — any logged-in user could overwrite any team's Devpost link or demo video. Both now lazy-import api.submissions.submissions_service.self_serve_team_edit, which runs the same _authorize_team_write gate as every other self-serve write in this feature (403 non-member, 409 closed-window unless admin). api.teams.teams_service itself must never import api.submissions (keep the dependency one-directional) — the import lives in the view function, not the service module.

Hardening from the PR #288 review (Sep 2026) — invariants, don't regress

  • save_project is the autosave hot path (~1.5s debounce): no send_slack_audit (blocking, no timeout) and no cache flush except the first legacy→draft save. Audit/flush belongs on submit_project only.
  • reminders_sent.by is "admin"/"cron", never a PropelAuth id — the hackathon doc is served by public endpoints; _PRIVATE_HACKATHON_FIELDS (services/hackathons_service.py) strips reminders_sent from get_single_hackathon_event and _process_hackathon_docs. Add any new operational field there. The cron returns HTTP 500 when any event fails (so the GitHub job goes red) after still processing the rest.
  • sanitize_markdown scrubs on*=/href=/src= only inside HTML-tag spans (_MARKDOWN_HTML_TAG_RE) — applying them to the whole string corrupts prose/code (const onSubmit = ...).
  • /api/github/activity is unauthenticated → private repos raise PrivateRepoError → same 404 as not-found.
  • teams/<id>/ CDN prefix is only trustworthy because POST /api/messages/upload-image calls authorize_team_upload_directory (members/admin only; ../bare teams → 400). Keep that gate if the upload route is ever rewritten.
  • CI's "Tests" job does not run pytest (commented out in main.yml) — run the suites locally, per directory (api/messages/tests files individually).

Documented-only findings (not fixed — out of scope or not trivial)

  • Legacy save_team (services/teams_service.py, backing the old POST /api/messages/team — the route's own docstring says "kept for backward compatibility, new teams should use /api/team/queue") calls create_github_repo with an outdated positional argument order/count (missing org_name/devpost_url, has two extra params the current signature doesn't take). Every call to this legacy path already fails. Not fixed here: the correct fix needs a hackathon-event lookup this function doesn't currently do, and the path appears to have no live callers left — a real fix belongs with a decision about whether to delete the legacy route instead of repairing it.
  • hackathon.devpost_url is read in services/volunteers_service.py but save_hackathon never writes a top-level devpost_url field on the hackathon doc — it's links[]-only. Not a bug introduced or fixed here; DevPost is now optional/deprecated for the team dashboard anyway, so links[] stays the source of truth.
  • GET /api/hacker/applications/<event_id> (api/volunteers/volunteers_views.py, public, @auth.optional_user) exposes user_id and isSelected for every hacker applicant via get_all_hackers_by_event_id. findteam.js matchmaking on the frontend depends on user_id being present, so this isn't trivially fixable without a frontend change too — a lean projection (drop user_id/isSelected from the public shape, resolve matching some other way) is a good follow-up but out of scope here.
  • Judge-page field naming split: some frontend judge-page code reads devpost_url/video_url while the team doc itself uses devpost_link/demo_video_url. The judging API now fills both demo_video_url and video_url (see above) so either naming convention on the frontend keeps working; a follow-up could rename one side for consistency but isn't required.

Per-worker caches vs. get_team (Sep 2026)

Gunicorn runs --workers 2, and every TTLCache in this codebase is per-process. clear_all_caches() / a service's clear_cache() only busts the worker that handled the write — the other worker keeps serving stale data for the TTL. services/teams_service.py::get_team is therefore no longer @cached whole: the team doc is ONE Firestore get and is read fresh on every call; only the member-profile fan-out (db.get_all over users) is cached (_TEAM_USERS_CACHE, keyed by the tuple of member ids, 10 min). That is what made the public team page pick up a project-story edit immediately instead of "sometimes, within 10 minutes". doc_to_json (common/utils/firestore_helpers.py) converts a DocumentSnapshot directly and never caches it (the read already happened, so fresh data always wins); only DocumentReference inputs go through its per-process cache (keyed by doc id, 10-min TTL), which is why get_single_hackathon_id/get_single_npo/get_npo_by_hackathon_id pass doc.get() snapshots. Every dict/list it returns is a fresh copy (_copy_json_like — recursive dict/list copy, never deepcopy, since DocumentReference leaves hold the client), so callers may mutate the result (e.g. pop('project_story')) without corrupting other callers. The event payload (get_single_hackathon_event, 10-min TTL) still has this cross-worker staleness for gallery/results views — accepted for now.

CDN URL prefix contract (Sep 2026)

common/utils/cdn.py::cdn_server() is the ONE way to get the public CDN origin: read at call time (not import), rstrip("/"), default https://cdn.ohack.dev. upload_to_cdn / generate_signed_upload_url build URLs with it and api/submissions/submissions_service.py::_cdn_server delegates to it, so a producer and a validator can never disagree on the prefix (the "Thumbnail: must be an ohack CDN URL under teams//" bug — a trailing slash or unset CDN_SERVER on one side). _validate_own_cdn_image logs the rejected URL + expected prefix at WARNING so the next mismatch is diagnosable from Render logs. api/jobs and users_service still carry their own os.getenv("CDN_SERVER", …) copies — migrate them to cdn_server() when touched.

Hardening stack (Sep 2026) — PR-A security

Plan + evidence live in the frontend repo: frontend-ohack.dev/docs/plans/hardening-security-seo-reliability-2026-09.md. Every change below landed test-first; the tests are the contract.

  • @bp.route must be the OUTERMOST decorator. Two routes (GET /hackathon/<event>/<type>/checkins, PATCH /api/problem-statements/events) had @auth.* above @bp.route, so Flask registered the undecorated function and served volunteer PII (email, phone) to anyone. test/common/test_view_decorator_order.py AST-scans every api/**/*_views.py and fails the build on a repeat. Route-level proof uses the rejecting auth stub in test/common/auth_stubs.py (rejecting_auth_module: no Authorization → 401, no X-Org-Id → 403) — the pass-through stub can't detect a dropped decorator.
  • Newsletter routes are volunteer.admin-gated (they were an open Gmail relay). POST /<subscribe>/<doc_id> stays require_user on purpose (per-user self-service). The module imports common.auth like every other view (its own init_auth made it un-importable under test).
  • Hackathon requests (public capability-link flow, /hack/request/<id>): update_hackathon_request filters the body to HACKATHON_REQUEST_EDITABLE_FIELDS (= the frontend HackathonRequestForm formData keys; lockstep test in api/messages/tests/test_hackathon_requests.py::FRONTEND_FORM_KEYS), 404s before any email when the doc is missing, and confirms to the STORED contactEmail (never the body's). The public GET strips adminNotes. New ids are uuid4. Admin edits use the separate admin_update_hackathon_request.
  • Uploads (POST /api/messages/upload-image): one gate, api/submissions/submissions_service.py::authorize_upload_directory — teams/<id>/… delegates to the #289 membership gate; non-admins may otherwise only write single-segment USER_UPLOAD_DIRECTORIES (images hackers volunteers mentors judges sponsors uploads) or hackathons/<event>/planning/… when hackathon_planning_service.can_write_plan_for_event says they edit that plan; everything else (site assets, event galleries, nonprofit logos, blog media) is admin-only (403 directory_not_allowed); ../odd characters → 400 invalid_directory. Non-admins can't overwrite: upload_image_to_cdn(request, allow_overwrite=False) answers 409 file_exists (multipart path; cdn.blob_exists — upload_to_cdn itself still overwrites because certificates/hearts/openai rely on it). MAX_CONTENT_LENGTH = 32 MiB with a JSON 413 (api/exception_views.py) because the app forces JSON content-type and frontend callers do res.json().
  • Hacker directory (GET /api/hacker/applications/<event>) is require_user and projected to HACKER_DIRECTORY_FIELDS (services/volunteers_service.py) — every key findteam.js reads + teamCode + isSelected (peer votes). Add to the allowlist if the finder needs a new field; phone/deposit/sent_emails must never ship.
  • Team payloads: services/teams_service.py::public_team_view strips PUBLIC_TEAM_STRIPPED_FIELDS (admin_notes nonprofit_rankings comments communication_history) in get_team, get_teams_list, get_teams_batch, get_single_hackathon_event, /me, and (non-admins only) GET /api/team/<hackathon_id> whose member docs are also trimmed to {id,user_id,name,nickname,profile_image} (full user docs incl. email were shipping to any logged-in user). Admins keep full payloads there; the full single doc is GET /api/team/admin/<teamid> (get_team_admin), which the frontend's adminTeamApi prefers (404 → public fallback). mentor_* fields are public BY DESIGN — don't strip them.
  • Deposit webhook: _handle_checkout_session_completed compares amount_total with the event's default_amount_cents BEFORE the already-paid shortcut; short → deposit_status="underpaid" + Slack audit, never paid; missing config/lookup error fails open. Best-effort only (the submit path still accepts deposit fields by design); PaymentIntent verification is the follow-up.
  • Shared-secret headers (BACKEND_NEWS_TOKEN, BACKEND_PRAISE_TOKEN, store X-Webhook-Secret) compare with hmac.compare_digest and never match when the env var is unset. GET /news?limit= → 400 on non-int, clamped to 1..200.
  • Running tests locally: ENVIRONMENT=test SLACK_WEBHOOK= SLACK_BOT_TOKEN= RESEND_API_KEY= FIREBASE_CERT_CONFIG='<structurally valid throwaway service-account JSON>' python -m pytest <dir> — Slack vars EMPTY (dummies make send_slack_audit raise on requests.post), and common/utils/firebase.py builds credentials.Certificate at import, so .env.example's placeholder cert blocks collection of almost every suite. api/messages/tests files run individually. api/certificates/tests hangs on network — exclude it from gated runs.

Hardening stack (Sep 2026) — PR-B reliability

  • doc_to_json semantics are in "Per-worker caches vs. get_team" above: snapshots bypass the cache, references cache 10 min, every return is a copy. _enrich_teams_users_batch never blanks an already-enriched team; api/judging/judging_service.py resolves users[] entries that are dicts (u["id"]) — judges' team pages showed no members before. Tests: test/common/utils/test_firestore_helpers_cache.py, api/messages/tests/test_enrich_teams_batch.py, api/judging/tests/test_team_members_shape.py.
  • Rate limits (ratelimit @limits) are per-process and all-clients-combined. A tripped limit is now HTTP 429 {"error":"rate_limited"} + Retry-After (api/exception_views.py), not a 500. @cached must sit OUTERMOST above @limits so cache hits don't count (get_npo_list had it reversed → /api/messages/npos 500'd everyone after 20 req/min). Views wrapped in a blanket except Exception (volunteers/planning/store) still turn it into their own 500 — follow-up. No per-IP limiter yet: a next build fires ~550 requests from one IP.
  • GET /api/health (api/health/health_views.py, zero imports beyond Flask) backs [[http_service.checks]] in fly.toml.
  • CI runs pytest again (.github/workflows/main.yml tests job, Python 3.10): a throwaway service-account JSON is generated with openssl into FIREBASE_CERT_CONFIG at job time (never commit one), Slack/Resend/PropelAuth vars are EMPTY (OPENAI_API_KEY is a DUMMY non-empty string — the OpenAI client is constructed at import and rejects an empty key), and the loop runs the green set (api/{broadcasts,judging,peer_votes,praisebot,problemstatements,submissions,surveys,teams,users,volunteers,health}/tests, test/common, test/services, plus api/messages/tests/test_*.py one file at a time). Excluded with reasons in the workflow: contact (real signature bug + recaptcha env), github/slack (init_auth at import), leaderboard (2 known), certificates (network hang). Add new test dirs to that loop; deploy needs [lint, tests], so a red suite now blocks the deploy.