Skip to content

listings + housing: the OpenListing directory, and Redfin's open market data - #144

Merged
ralyodio merged 1 commit into
mainfrom
worktree-nichedb-listings-housing
Sep 25, 2026
Merged

ralyodio merged 1 commit into
mainfrom
worktree-nichedb-listings-housing

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

Two adapters. One reads the spec published on logicsrc today; the other pulls the closest thing to open MLS data that exists.

openlisting — the reference directory

A listing is believed only when served from the origin its id claims — the rule OpenWebring established here. Otherwise anyone could serve a document claiming to be someone else's listing and this would carry it. A seller's index may only point at the seller's own listings, checked the same way.

Three decisions follow the spec rather than convenience:

  • Prices stay decimal strings end to end. 2450.10 isn't representable in binary floating point, and a listing is a price. A test asserts the built item never turns it into a number.
  • A street is dropped unless the seller said precision: exact or street — so a landlord publishing a postal code for an occupied home stays published at a postal code.
  • bedrooms: 0 survives, because a studio is not an unstated bedroom count. The one documented place absent and zero differ.

Closed listings are ingested and tagged, not discarded. The spec asks sellers to keep a sold listing served 90 days with closed_at, because a directory learning from a 404 can't tell "sold" from "the server is down" — and the outcome is the part every listing site loses. There's a feed for exactly that.

It ships no default source. The spec published today and nobody serves a descriptor yet; a seeded source pointing at an invented origin would fail every run and read as the adapter being broken.

redfin-market — what the market actually did

The open S3 bucket Redfin publishes: median sale/list price, price per sq ft, homes sold, pending sales, new listings, inventory, months of supply, days on market, sale-to-list — monthly since 2012, by property type.

Six levels, measured 2026-09-25:

level size default
neighborhood 2.24 GB opt-in
zip 1.48 GB opt-in
city 955 MB opt-in
county 230 MB ✅
state 8.6 MB ✅
national 0.5 MB ✅

Redfin is a brokerage with MLS access giving away the derived view — and that view is the half a person actually asks for. The listings themselves are not open and won't become open: 484 separate MLSs, each needing a licence, a broker and a signed agreement, none redistributable.

It is not a listing, and every row says so in data.note. OpenListing forbids publishing aggregate market data as a listing — no seller, no offer — so this is housing-market in its own collection, deliberately apart from /c/listings.

Two collisions caught while wiring up

  • housing already exists, with six adapters (Land Registry sold prices, Eurostat HPI, permits, Freddie Mac rates). Redfin joins it and the existing description is extended, rather than creating a second collection with the same slug.
  • The registry test's hardcoded collection list needed listings — the same check that caught parts earlier today.

Failing loudly

Both header maps throw rather than degrade: a Redfin file missing a column it reads fails instead of producing rows where every measure is absent. And an empty TSV cell is null, never 0 — a ZIP with no sales isn't a ZIP where the median price was zero, and charting it as zero puts a spike through every quiet market.

Checks

59 new tests (built on a real header and row from state_market_tracker), 2,101 across the suite, lint clean. The usual 5 Redis/Postgres integration failures this dev box can't run.

🤖 Generated with Claude Code

…et data

Two adapters. One reads the spec published on logicsrc today; the other
pulls the closest thing to open MLS data that exists.

**openlisting** is the reference directory the spec names. A listing is
believed only when served from the origin its `id` claims -- the rule
OpenWebring established here -- because otherwise anyone could serve a
document claiming to be somebody else's listing and this would carry it.
A seller's index may only point at the seller's own listings, checked
the same way.

Three decisions follow the spec rather than convenience. Prices stay
DECIMAL STRINGS end to end: 2450.10 is not representable in binary
floating point, a listing is a price, and a test asserts the built item
never turns it into a number. A street is dropped unless the seller
said the precision was `exact` or `street`, so a landlord publishing a
postal code for an occupied home stays published at a postal code. And
`bedrooms: 0` survives, because a studio is not an unstated bedroom
count -- the one documented place where absent and zero differ.

Closed listings are ingested and tagged, not discarded. The spec asks
sellers to keep a sold listing served ninety days with `closed_at`
because a directory learning from a 404 cannot tell "sold" from "the
server is down", and the outcome is the part every listing site loses.
There is a feed for exactly that.

It ships NO default source. The spec published today and nobody serves
a descriptor yet; a seeded source pointing at an invented origin would
fail every run and read as the adapter being broken.

**redfin-market** walks the open S3 bucket Redfin publishes: median sale
and list price, price per square foot, homes sold, pending sales, new
listings, inventory, months of supply, days on market and sale-to-list,
monthly since 2012, by property type. Six region levels, measured
2026-09-25: neighborhood 2.24 GB, zip 1.48 GB, city 955 MB, county 230
MB, state 8.6 MB, national 0.5 MB. The two enormous ones are opt-in.

Redfin is a brokerage with MLS access giving away the derived view, and
that view is the half a person actually asks for. The listings
themselves are not open and will not become open: 484 separate MLSs,
each needing a licence, a broker and a signed agreement, none
redistributable.

It is NOT a listing, and every row says so in `data.note`. OpenListing
forbids publishing aggregate market data as a listing -- it has no
seller and no offer -- so this is `housing-market` in its own
collection, deliberately apart from /c/listings.

Two collisions caught while wiring it up. `housing` ALREADY EXISTS, with
six adapters (Land Registry sold prices, Eurostat HPI, permits, Freddie
Mac rates); Redfin joins it and the existing description is extended,
rather than a second collection with the same slug. And the registry
test's hardcoded collection list needed `listings`, the same check that
caught `parts` earlier today.

Both header maps fail loudly rather than silently: a Redfin file missing
a column it reads throws instead of producing rows where every measure
is absent. An empty TSV cell is null, never zero -- a ZIP with no sales
is not a ZIP where the median price was zero, and charting it as zero
puts a spike through every quiet market.

59 new tests, 2101 across the suite, lint clean. The usual 5
Redis/Postgres integration failures this box cannot run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ralyodio
ralyodio marked this pull request as ready for review September 25, 2026 10:33
@ralyodio
ralyodio merged commit 7e7ae43 into main Sep 25, 2026
3 checks passed
@ralyodio ralyodio mentioned this pull request Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant