feat(catalog): add "What's in the Catalog?" discovery tutorial - #67
Conversation
The discovery counterpart to the two paid catalog demos: instead of an agent that *pays*, it shows how you *find* what's in the catalog — the same live Nevermined catalog read three ways (by eye, by an agent, by a crawler) over public, unauthenticated, free endpoints. Because discovery is free and read-only, its "See it run" panel is genuinely functional in the browser (a new `discover` tier), unlike the watch-only `recap` demos: - catalog/discover-the-catalog/ — the raw tutorial (README, run-demo.sh, demo-prompt.txt, the 57s video + EN/ES subtitles) and the standalone zero-dependency playground kept as the runnable reference. - showcase: a new `DiscoverPanel` + `/api/catalog` same-origin proxy that forwards catalog/ARD/MCP calls server-side (the catalog CORS only reflects *.nevermined.app origins). Proxies only the fixed Nevermined upstreams — no open relay. Pure request/parse logic lives in lib/catalog-discovery.mjs with a `node` self-check. - Registered in content/tutorials.ts with the written explanation + video. The `discover` tier/kind is additive: existing live/recap tutorials render unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk
|
👀 Reviewing |
r-marques
left a comment
There was a problem hiding this comment.
🤖 Automated PR review — 🟡 Mergeable with nits
Adds the "What's in the Catalog?" discovery tutorial: a standalone playground, a same-origin proxy route in the showcase, and a new recap video. The security-shaped part — two new server-side proxies — is the part I checked hardest, and both hold up. One real gap: the new video will be missing from the deployed image, and it works locally, which is exactly what hides it.
What I reviewed
- Diff range
origin/main...HEAD— 23 files, +1,951/−5, headed7778eb. - Panel: none — stood on direct verification.
Verified
- The showcase proxy's SSRF claim is accurate, not just asserted.
app/api/catalog/route.tssays the client picks anopfrom a fixed set and the URL is built server-side; I checked every builder inlib/catalog-discovery.mjs.restRequestputs the term throughnew URLSearchParams(encoded) andkeyword()has already stripped it to[a-z0-9\s];ardRequest/mcpRequest/exploreRequestall send text in the body against hardcoded paths;normalizeRailsis a strictx402|mppwhitelist, so evenrails[0]can't inject. No client-controlled data reaches the host or path. - The playground proxy forwards no client headers.
proxy()constructs a fresh header object (accept, pluscontent-typeon writes) rather than passingreq.headersthrough, so there's noAuthorizationpassthrough;pcomes fromnew URL(...).pathname, which is already normalised, so..can't escape the API host.serveStaticdocuments and enforces its own containment. - No credentials anywhere in the new route, discovery lib, playground or demo scripts — discovery is genuinely unauthenticated, as the comments say.
- My round-3 nit on the SRT→VTT conversion was taken exactly, and it works:
sed 's/\r$//; /-->/ s/,\([0-9][0-9][0-9]\)/.\1/g'is now anchored to cue lines. I regenerated both new.vttwith it — byte-identical to the committed files. - The
verifyjob added in #65 covers this PR — it triggers onshowcase/**, andnext buildtype-checks the new route,DiscoverPaneland the expandedtutorials.ts. That is the CI gap from two rounds ago now doing its job.
Verdict: 🟡 Mergeable with nits — 0 blockers, 1 should-fix, 1 nit.
🧭 Inline comments (2)
Each is posted on its line in Files changed.
- 🟡 SHOULD FIX — The new recap video won't be in the deployed image — the Dockerfile copies mp4s by name and wasn't updated —
showcase/content/tutorials.ts:626 - 💡 NIT —
listen(PORT)binds every interface while the README promises localhost —catalog/discover-the-catalog/playground/server.mjs:112
r-marques
left a comment
There was a problem hiding this comment.
🟡 Approved with comments — no blockers; 1 should-fix, 1 nit(s) left inline. Auto-merge was disabled so these can be looked at first.
Inline review: #67 (review)
…o localhost Address review nits on PR #67: - showcase/Dockerfile: the recap/discover videos are gitignored under showcase/public and copied by name from catalog/; add the discover-the-catalog mp4 COPY line. Without it the image builds green but the tutorial's <video> 404s in the deployed image (it only resolved locally via sync-media.sh). - playground/server.mjs: default-bind to 127.0.0.1 (was 0.0.0.0) so the teaching artifact matches its own "http://localhost" banner/README; HOST env still allows 0.0.0.0 for container/VM deploys. README documents it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk
Why this matters
The two existing Catalog demos show an agent spending money — finding services and paying for them autonomously. But before anything can be paid for, it has to be found. This tutorial teaches that first half: how a person, an agent, or a crawler discovers what's in the Nevermined Catalog — every discovery endpoint being public, unauthenticated, and free. Because it's read-only and costs nothing, its "See it run" panel isn't a watch-only video like the paid demos — it actually runs live in the browser against the real catalog (~160 agents today), so a visitor can type a question and watch it answered three ways in real time. It's the natural on-ramp that sits in front of the paid demos.
What this adds
A single new Catalog tutorial, "What's in the Catalog?" (slug
discover-the-catalog):catalog/discover-the-catalog/— the raw tutorial content, mirroring the two sibling catalog folders: the layeredREADME.md(every curl/MCP/ARD example verified live),run-demo.sh(a free, no-key terminal tour),demo-prompt.txt, the 57s video +.en/.es.srt, and the standalone zero-dependencyplayground/kept as the runnable "run it locally" reference.content/tutorials.tswith the written explanation (learn · how · under-the-hood, sourced from the README) and the guided-tour video (.vttsubtitles derived from the.srt).components/DiscoverPanel.tsxruns the "one question, three altitudes" discovery live: human result cards (by eye), the exact MCP/ARD/REST request + JSON an agent makes (by an agent), the/.well-known/ard.jsonfeed entry (by a crawler), and a live ARD/explorefacet histogram of the rails/types/tags behind the answer.app/api/catalog/route.tsforwards the catalog/ARD/MCP calls server-side. This is required: the catalog API's CORS only reflects*.nevermined.apporigins, so a browser calling it from the showcase's origin is blocked (exactly what the standaloneplayground/server.mjssolves). It is SSRF-safe: the client picks anopfrom a fixed set; the upstream host + path are built entirely server-side, never from client input, so it can't be turned into an open relay. Pure request-building/parsing logic lives inlib/catalog-discovery.mjswith anodeself-check.Design decision (the one that was mine to make)
How should a functional discovery demo fit the showcase model? The
livetier's/api/agent+demo-agent.mjsis built for the x402 payment handshake (402 → authorize → settle), which discovery has none of.recapis watch-only. I added a third tier + run variant,discover(option (b) from the brief), with its own panel and/api/catalogproxy referenced from the entry.This was chosen as the least-invasive option that keeps the shared page model intact for every other tutorial. The touches to shared files (
lib/types.ts,app/t/[slug]/page.tsx,app/globals.css,app/page.tsxbadge) are strictly additive and gated on=== "discover"— they add a new discriminated-union member and new render branches; they do not change how any existing tutorial renders. Verified in a browser:song-from-the-headlinesstill renders "recap · watch it run" with its video + receipt, andlangchain-research-agentstill renders its live payment panel — neither shows the discovery panel. Because this is purely additive and was explicitly delegated in the brief ("referenced from the entry"), it is not a reshaping of the shared model, so I proceeded rather than escalating.Rejected: (a) bolting a "discovery mode" onto the payment-shaped
liveinfrastructure — it would muddy the x402 sandbox with an unrelated concern; and reusingrecap— it would mislabel a genuinely interactive, free demo as "watch-only", which is the opposite of the tutorial's point.Scope
Touches only the new tutorial plus the minimal additive wiring:
catalog/discover-the-catalog/, its showcase entry, its media underpublic/media/discover-the-catalog/, the one addedcopyline inscripts/sync-media.sh, the new panel + proxy route, and the additivediscovertier/kind in the shared type/page/CSS. No other tutorial, the two paid catalog demos, or shared component behavior was modified. The large.mp4is committed undercatalog/and gitignored underpublic/media/(pulled in bynpm run sync:media), matching the sibling demos.Test plan
node lib/demo-agent.mjs— existing sandbox self-check still passes.node lib/catalog-discovery.mjs— new self-check for keyword extraction, rail filters, ARD/explore body shape, SSE parsing, MCP unwrap, and feed matching.npm run build— green; type-checkscontent/tutorials.tsand generates all 11 tutorial pages.npm run sync:media— copies the video + derives.vttfrom.srtfor the new slug.npm run dev, headless Chromium) against the production catalog: the readout populated (162 agents · 13 categories · 38 x402 / 124 MPP), the hero question returned real cards, all three altitudes populated, the feed entry matched, and the/explorebars filled. Clicking a preset ("weather") re-queried live and refreshed all panes. Confirmed the two existing catalog/live tutorials render unchanged.🤖 Generated with Claude Code
https://claude.ai/code/session_01NJVmLG9aiAzymDX4n56duk