Skip to content

Publish text.redact-entities@1.0.0 (PII redaction via NER reuse) - #482

Merged
enricopiovesan merged 1 commit into
mainfrom
publish/text-redact-entities-1.0.0
Sep 12, 2026
Merged

enricopiovesan merged 1 commit into
mainfrom
publish/text-redact-entities-1.0.0

Conversation

@enricopiovesan

Copy link
Copy Markdown
Contributor

Summary

Publishes text.redact-entities@1.0.0 -- PII redaction (PER/ORG/LOC/MISC bracketed placeholders) reusing text.detect-entities' (#465) real DistilBERT-NER forward pass directly. Closes #481, the second pick from the post-#473 "Five Engines, What Next" downstream-capability review, after audio.transcribe-speech@1.1.0's VAD gate (#477).

Why a new capability, not a version bump

Unlike the VAD-gate case (a version bump because "transcribe, robustly" was still the same verb), "redact" is a genuinely different action from "detect" for a caller -- different intent, different output shape (redacted_text vs. raw entities). This is a new capability id.

Shape

  • capability-src/text-detect-entities gets the same lib/bin split proven on audio-detect-speech-segments for audio.transcribe-speech@1.1.0 — gate Whisper on a Silero VAD speech-presence pre-check #477: engine logic (Model, encode, forward_logits, decode_entities, Entity, plus an always-available fixture_model() for dependents' own tests) moves to a new lib.rs; main.rs becomes a thin _start shim. text.detect-entities@1.0.0's published contract/artifact is untouched -- source-only refactor, all 19 existing tests pass unchanged.
  • capability-src/text-redact-entities takes a path dependency on that lib, runs the exact same encode/forward/decode pipeline, then substitutes a bracketed placeholder ([PER]/[ORG]/[LOC]/[MISC]) for each entity span instead of returning raw entity data.
  • No new model, no new license/verification burden -- same weight table, same dslim/distilbert-NER (Apache-2.0) license record as text.detect-entities.

Verification

  • Verified against the real published model via the compiled wasm32-unknown-unknown --features full-model artifact under wasmtime on real sentences: "Sarah Chen from Acme Corp will follow up with the Berlin team next week." redacts to "[PER] from [ORG][ORG][ORG] will follow up with the [LOC] team next week.", and "Elon Musk founded SpaceX in California." redacts to "[PER][PER] founded [ORG][ORG] in [LOC]." -- both correctly reproducing text.detect-entities' own documented BIO-fragmentation behavior on these exact sentences (verified directly, not assumed to carry over).
  • capability-src/text-redact-entities: 10 #[test] cases (placeholder mapping, single/multi-span substitution, no-entities passthrough, defensive out-of-order-span handling, full-pipeline determinism, missing-text-field default); cargo llvm-cov functions=100%, lines=100%, regions=100%.
  • capability-src/text-detect-entities: all 19 pre-existing tests still pass post-refactor.

Governing Spec

  • 001-registry-foundation
  • 014-extraction-compatibility
  • 017-persona-registry
  • 018-capability-test-coverage

Project Item

https://github.com/orgs/traverse-framework/projects/3 -- #481

Definition of Done

  • capability-src/text-redact-entities/ #![no_std]; ABI-clean (imports exactly {fd_read, fd_write, proc_exit}); #[test] coverage functions=100%/lines&regions>=95%.
  • capabilities/text/text.redact-entities/1.0.0/contract.json -- full I/O schema, ai.model_backed: true, spec-024 risk.determinism_class: model_derived, spec-023 authoring, provenance citing text.redact-entities@1.0.0 — PII redaction reusing the DistilBERT-NER engine #481.
  • text.detect-entities@1.0.0 contract/artifact untouched by the lib/bin source refactor.
  • New persona privacy-export-reviewer scaffolded via scripts/scaffold/new-persona.sh (no existing persona fit).
  • Signed .wasm, pre_pr_check.sh green, human merge.

Validation

  • bash scripts/ci/pre_pr_check.sh <this body> with BASE/HEAD wired as CI wires them.

🤖 Generated with Claude Code

Second pick from the post-#473 "Five Engines, What Next" downstream-
capability review, after audio.transcribe-speech@1.1.0's VAD gate
(#477). Reuses text.detect-entities' (#465) real DistilBERT-NER forward
pass directly -- no new model, no new license/verification burden.

New capability id, not a version bump: "redact" is a genuinely different
action from "detect" for a caller (different intent, different output
shape), unlike the VAD case where "transcribe, robustly" was still the
same verb.

capability-src/text-detect-entities gets the same lib/bin split proven
on audio-detect-speech-segments for #477: engine logic (Model, encode,
forward_logits, decode_entities, Entity, plus an always-available
fixture_model() for dependents' own tests) moves to a new lib.rs, main.rs
becomes a thin _start shim. text.detect-entities@1.0.0's published
contract/artifact is untouched -- source-only refactor, all 19 existing
tests pass unchanged.

text-redact-entities takes a path dependency on that lib, runs the exact
same encode/forward/decode pipeline, then substitutes a bracketed
placeholder ([PER]/[ORG]/[LOC]/[MISC]) for each entity span instead of
returning raw entity data.

Verified against the real published model via the compiled wasm artifact
under wasmtime: "Sarah Chen from Acme Corp will follow up with the
Berlin team next week." redacts to "[PER] from [ORG][ORG][ORG] will
follow up with the [LOC] team next week." -- correctly reproducing
text.detect-entities' own documented BIO-fragmentation behavior on this
exact sentence, verified directly rather than assumed to carry over.

capability-src/text-redact-entities: 10 #[test] cases. cargo llvm-cov:
functions=100%, lines=100%, regions=100%.

New persona privacy-export-reviewer scaffolded (no existing persona fit
"blank PII before text leaves a privacy boundary").

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@enricopiovesan
enricopiovesan enabled auto-merge (squash) September 12, 2026 03:01
@enricopiovesan
enricopiovesan merged commit a304533 into main Sep 12, 2026
11 checks passed
@enricopiovesan
enricopiovesan deleted the publish/text-redact-entities-1.0.0 branch September 12, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

text.redact-entities@1.0.0 — PII redaction reusing the DistilBERT-NER engine

1 participant