listings + housing: the OpenListing directory, and Redfin's open market data - #144
Merged
Merged
Conversation
…et data Two adapters. One reads the spec published on logicsrc today; the other pulls the closest thing to open MLS data that exists. **openlisting** is the reference directory the spec names. A listing is believed only when served from the origin its `id` claims -- the rule OpenWebring established here -- because otherwise anyone could serve a document claiming to be somebody else's listing and this would carry it. A seller's index may only point at the seller's own listings, checked the same way. Three decisions follow the spec rather than convenience. Prices stay DECIMAL STRINGS end to end: 2450.10 is not representable in binary floating point, a listing is a price, and a test asserts the built item never turns it into a number. A street is dropped unless the seller said the precision was `exact` or `street`, so a landlord publishing a postal code for an occupied home stays published at a postal code. And `bedrooms: 0` survives, because a studio is not an unstated bedroom count -- the one documented place where absent and zero differ. Closed listings are ingested and tagged, not discarded. The spec asks sellers to keep a sold listing served ninety days with `closed_at` because a directory learning from a 404 cannot tell "sold" from "the server is down", and the outcome is the part every listing site loses. There is a feed for exactly that. It ships NO default source. The spec published today and nobody serves a descriptor yet; a seeded source pointing at an invented origin would fail every run and read as the adapter being broken. **redfin-market** walks the open S3 bucket Redfin publishes: median sale and list price, price per square foot, homes sold, pending sales, new listings, inventory, months of supply, days on market and sale-to-list, monthly since 2012, by property type. Six region levels, measured 2026-09-25: neighborhood 2.24 GB, zip 1.48 GB, city 955 MB, county 230 MB, state 8.6 MB, national 0.5 MB. The two enormous ones are opt-in. Redfin is a brokerage with MLS access giving away the derived view, and that view is the half a person actually asks for. The listings themselves are not open and will not become open: 484 separate MLSs, each needing a licence, a broker and a signed agreement, none redistributable. It is NOT a listing, and every row says so in `data.note`. OpenListing forbids publishing aggregate market data as a listing -- it has no seller and no offer -- so this is `housing-market` in its own collection, deliberately apart from /c/listings. Two collisions caught while wiring it up. `housing` ALREADY EXISTS, with six adapters (Land Registry sold prices, Eurostat HPI, permits, Freddie Mac rates); Redfin joins it and the existing description is extended, rather than a second collection with the same slug. And the registry test's hardcoded collection list needed `listings`, the same check that caught `parts` earlier today. Both header maps fail loudly rather than silently: a Redfin file missing a column it reads throws instead of producing rows where every measure is absent. An empty TSV cell is null, never zero -- a ZIP with no sales is not a ZIP where the median price was zero, and charting it as zero puts a spike through every quiet market. 59 new tests, 2101 across the suite, lint clean. The usual 5 Redis/Postgres integration failures this box cannot run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two adapters. One reads the spec published on logicsrc today; the other pulls the closest thing to open MLS data that exists.
openlisting— the reference directoryA listing is believed only when served from the origin its
idclaims — the rule OpenWebring established here. Otherwise anyone could serve a document claiming to be someone else's listing and this would carry it. A seller's index may only point at the seller's own listings, checked the same way.Three decisions follow the spec rather than convenience:
2450.10isn't representable in binary floating point, and a listing is a price. A test asserts the built item never turns it into a number.precision: exactorstreet— so a landlord publishing a postal code for an occupied home stays published at a postal code.bedrooms: 0survives, because a studio is not an unstated bedroom count. The one documented place absent and zero differ.Closed listings are ingested and tagged, not discarded. The spec asks sellers to keep a sold listing served 90 days with
closed_at, because a directory learning from a 404 can't tell "sold" from "the server is down" — and the outcome is the part every listing site loses. There's a feed for exactly that.It ships no default source. The spec published today and nobody serves a descriptor yet; a seeded source pointing at an invented origin would fail every run and read as the adapter being broken.
redfin-market— what the market actually didThe open S3 bucket Redfin publishes: median sale/list price, price per sq ft, homes sold, pending sales, new listings, inventory, months of supply, days on market, sale-to-list — monthly since 2012, by property type.
Six levels, measured 2026-09-25:
Redfin is a brokerage with MLS access giving away the derived view — and that view is the half a person actually asks for. The listings themselves are not open and won't become open: 484 separate MLSs, each needing a licence, a broker and a signed agreement, none redistributable.
It is not a listing, and every row says so in
data.note. OpenListing forbids publishing aggregate market data as a listing — no seller, no offer — so this ishousing-marketin its own collection, deliberately apart from/c/listings.Two collisions caught while wiring up
housingalready exists, with six adapters (Land Registry sold prices, Eurostat HPI, permits, Freddie Mac rates). Redfin joins it and the existing description is extended, rather than creating a second collection with the same slug.listings— the same check that caughtpartsearlier today.Failing loudly
Both header maps throw rather than degrade: a Redfin file missing a column it reads fails instead of producing rows where every measure is absent. And an empty TSV cell is
null, never0— a ZIP with no sales isn't a ZIP where the median price was zero, and charting it as zero puts a spike through every quiet market.Checks
59 new tests (built on a real header and row from
state_market_tracker), 2,101 across the suite, lint clean. The usual 5 Redis/Postgres integration failures this dev box can't run.🤖 Generated with Claude Code