Are these the same thing? arche decides, shows the evidence, and keeps the receipt.
An open-source engine for deciding when messy records of people, places, organisations, and products refer to the same real-world entity, with evidence, not just similarity scores or identifiers.
pip install arche-coreYour records came from different systems, written by different people, at different times. A supplier list, a registry export, six PDFs, a partner's spreadsheet. The same hospital, cooperative, product or person appears in all of them under different names, with no shared identifier anywhere.
Nothing in those files says which rows are about the same thing. That fact lives in the world, not in the data, so somebody has to decide it, and a decision that cannot be explained later is not much use in a system that pays people, ships medicine, or files a regulatory statement.
Most tools give you a similarity score. arche gives you a decision, the evidence behind it, and permission to say I don't know.
| detect | Find the entities and identifying data in text, documents and records. |
| resolve | Decide which real-world thing each reference points at, and abstain when the evidence does not support a verdict. |
| protect | Apply the statute that governs the data, citing the section it came from. |
| attest | Sign the decision together with the evidence and the exact representation that produced it. |
from arche import detect_pii, deidentify
note = "Patient Casey Example (NIN 12345678901) called from 0803 555 7890."
for d in detect_pii(note, jurisdiction="NG", backend="basic"):
print(d.category, (d.start, d.end), d.regulatory_citation) # PII-2-NIN (27, 38) NDPA-2023 s.30, NIMC Act s.27
safe = deidentify(note, jurisdiction="NG", backend="basic")
print(safe.text) # Patient Casey Example (NIN [NIN]) called from PHONE_d3100c11.
print(safe.decision_id) # red:sha256:... -- deterministic; give it a ledger and it replaysThe statute decides what counts and what happens to it, and every span carries the section it fell under. Nothing leaves the process; the base install runs on CPU with no model, and arche-core[detect2] adds GLiNER2-PII as a proposer the validators and the statute still decide over. A tokenised copy is still comparable -- two masked notes about one person link on their tokens, without either side holding the value.
One engine, five calibrated packs. A pack is configuration and data, never a fork.
| Pack | For | The hard case it handles |
|---|---|---|
person |
people | Diallo and Jallow are one name; two Ibrahim Musa are two men |
place |
facilities, addresses, settlements | identical names 282 km apart are two hospitals |
organisation |
companies, cooperatives, unions | a site and the company operating it share a name and a coordinate |
product_electronics |
catalogue items | a rare model code identifies; Black T-Shirt does not |
artist |
creative works and performers | one act, many stage names and transliterations |
from arche.resolve import reconcile
result = reconcile(supplier_list, certifier_registry,
entity="organisation", id_field="id")
for edge in result["matches"]:
print(edge["decision"], edge["score"], edge["evidence"])Three answers, not two: same_entity, review, different. The middle one is the point. It is Fellegi and Sunter's third region from 1969, which most production systems discard because a review queue costs money, and discarding it is where systems start asserting things they have not earned.
Unstructured input is a first-class entry point, not a preprocessing step you bolt on.
from arche import resolve_documents
report = resolve_documents("statements/*.pdf")
print(report.table())That parses each file, detects the identifying data with the governing statute attached, builds a record per document, and resolves them against each other. Every decision carries the extraction that produced it: the hash of the input bytes, the parser and its version, the digest of the rendering its spans point into. Upgrade the parser next year, re-run, and you can tell whether the answer changed or only the machinery did.
A decision_id is an address, not a record. Hand a verb a ledger and the receipt is kept with the input it was made from, so the id becomes something you can look up, explain and run again.
import arche
from arche import deidentify
ledger = arche.attach("duckdb:///notes.duckdb") # a DuckDB file you own
note = "Patient Casey Example (NIN 12345678901) called from 0803 555 7890."
safe = deidentify(note, jurisdiction="NG", backend="basic", store=ledger)
print(safe.decision_id)
# red:sha256:31d8e1ac9880676a152f8884902489200dfabd6c5c0950cdfe7eb6a75dacf89f
again = ledger.replay(safe.decision_id)
print(again.reproduced) # True: this installation still makes that decision, byte for byte
print(again.now["text"]) # Patient Casey Example (NIN [NIN]) called from PHONE_d3100c11.
print(again.changed) # {} -- nothing that fed it has movedreproduced is stronger than "the same answer": it means no detector, statute pack or engine version behind the decision has moved, and when one has, changed names it. now carries what the decision produces today, so a replay gives back the artefact and not only a verdict on it: the masked copy for a redaction, the question for a place endpoint, the verdict for a match.
Note which direction that runs in. The ledger keeps the original and the masked copy is re-derived from it, because [NIN] does not contain a national id and PHONE_d3100c11 is a hash. That is what makes the copy safe to hand on, and the ledger the file you keep on your own disk. Without store=, nothing is kept: the id still prints and is still deterministic, so you can re-derive it from the same input and compare, but there is nothing to replay.
The same from the shell, and the same three verbs for a match, a batch edge or a delivery address:
arche redact --text "Patient Casey Example (NIN 12345678901) called from 0803 555 7890." \n --jurisdiction NG --store notes.duckdb
arche explain red:sha256:31d8e1ac... --store notes.duckdb # the spans, and the section each fell under
arche replay red:sha256:31d8e1ac... --store notes.duckdb # reproduced: True, and the copy againEvery decision has a reproducible decision_id, hashed over the rounded evidence and the pinned versions. No timestamp, no raw float. Same inputs, same id, byte for byte.
from arche.resolve.reconcile import sign_edges
from arche.sign import generate_keypair
signed = sign_edges(result, private_key=key.private_key, kid=key.did_key)A recipient checks two independent things: the signature, which establishes who issued it and that nothing changed since, and the recomputed id, which establishes that the id is the honest address of this evidence rather than one lifted from a more favourable decision. They fail differently, which is what makes the pair useful. See re-verify a decision.
Published whichever way it falls, with the caveats attached rather than in a footnote.
| What | Baseline | arche | Read it as |
|---|---|---|---|
| Leipzig DBLP–ACM | out of the box, precision 0.8500, 391 false merges | precision 0.9506, recall 0.9960 | Complete ground truth, so false merges are counted rather than estimated. The gain is not default: it needs one declared discriminator on year |
| Leipzig Abt-Buy (products) | name only, F1 0.3443 | F1 0.7883, precision 0.9707 | 62 false merges down to 22 |
| Organisation lane | token-sort F1 0.8898 | F1 0.9493 | False merges 21 → 4. Anglophone restaurant listings, so it says nothing about African organisation names |
| Name frequency (ablation) | the same engine with the frequency signal off: precision 0.162, 7,705 false merges | precision 0.946, 41 false merges | 1,114 real same-surname pairs from a public voter register. Recall is the cost: 48% on same-person pairs differing by a dropped middle name |
| Multilingual detection | Presidio 37/48 | 47/48 | Not re-runnable. The 48-case set is not in this repo and nothing here computes the number. Unverified until rebuilt |
The honest ledger, in full, is on Benchmarks, including the benchmarks that are too small, the abstention policy that is not yet precommitted, and the head-to-head against frontier models that has not been run.
The engine is general. The organisation frequency table is built from company registrations across 65 jurisdictions, the product work is benchmarked on US retail catalogues, and the place work runs on UK hospitals and Nigerian clinics alike.
What is unusual is where the defaults were tested first. Jaro-Winkler, the string comparator underneath most record linkage, pays a bonus for a shared prefix, because it was tuned on US Census surnames where clerical typos land at the end of a word. Diallo and Jallow are one Fula family name split by a colonial spelling border, and they share no prefix at all. That assumption fails identically on Arabic transliteration, on Cantonese romanisation, and on any register where one name has three spellings.
So: general-purpose entity resolution that ships its representation data, built by people who met the defaults first where the defaults break hardest. Six statute packs ship, covering NDPA (Nigeria), POPIA (South Africa), Kenya DPA, Ghana DPA, GDPR and HIPAA Safe Harbor, alongside a baseline floor that applies where no pack exists.
pip install arche-core # ~3 MB, CPU only, no ML dependencies
pip install "arche-core[doc]" # PDF, DOCX, PPTX, XLSX, HTML
pip install "arche-core[detect2]" # GLiNER2-PII and GLiNER 2.5 as proposers
pip install "arche-core[presidio]" # Microsoft Presidio integration
pip install "arche-core[resolve]" # Splink + DuckDB at scale
pip install "arche-core[service,mcp]" # `arche serve` over HTTP, `arche mcp` for an agentOr the container, with every parser and both models inside it and nothing fetched at runtime:
docker run --rm -p 8766:8766 ghcr.io/unpatterned-labs/arche-core # arche serve
docker run --rm -i ghcr.io/unpatterned-labs/arche-core mcp # the MCP server, stdiodeploy/ has a compose file that puts an auth proxy in front of both.
Runs offline by default. Nothing leaves the machine unless you configure a backend that does, and an EgressGuard has to be cleared before it can.
Warning
arche-core is pre-1.0. Suitable for research, prototyping, evaluation, benchmarking and contribution. APIs may change between minor releases. Do not use it against real personal data until you have completed your own legal, privacy and security review.
We mean this literally. Entity resolution has two halves. The mathematics of combining evidence is solved, has a founding paper from 1969, and has excellent free software in Splink. arche is about the other half: what the records look like when they are compared, what counts as agreement, and what agreement is worth. Every gain claimed here is a gain in representation, and none of it is a better estimator.
arche studio is a local reading tool. The standard library, no framework, and it is in the wheel: pip install arche-core is the whole install.
arche studio # opens http://127.0.0.1:8765Five modes over one evidence panel:
| Compare | two ad-hoc records, the decision, and a plain sentence saying why it held back |
| Spatial roles | paste text or drop a document; origins, destinations and waypoints marked with the cue that decided each |
| Redact | run a record through a statute pack and see the action, the cited section, and what was written instead |
| Verify | check a signed decision two ways: the signature, and whether its id recomputes from its own evidence |
| Review queue | a dense grid over an adjudication pack, sortable and searchable, marking outcomes to a labelled copy |
It is alpha, says so on the page, binds to 127.0.0.1, and has no authentication. It is a tool for reading our own queues, not a product.
| Five minutes | three commands, no data of your own |
| From Python | the five verbs, one example each |
| How it works | the decision, the evidence, the backends, the statutes |
| Compare two records | the verdict, its two fields, and why an identical pair can be review |
| Attested answers | a signed answer someone who does not trust you can verify |
| Runnable notebooks | 19, from a first call to reconciling a national register |
The representation data is open on purpose. A pack that everyone corrects is a better pack, and the corrections are the point. If you know an equivalence, a vocabulary or a frequency we do not, that is the highest-value contribution there is. See CONTRIBUTING.md.
Apache-2.0. The datasets carry their own licences, recorded beside them.