Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions assets/vendor/searchable-pdf/NotoSansKR-400.js

Large diffs are not rendered by default.

93 changes: 93 additions & 0 deletions assets/vendor/searchable-pdf/NotoSansKR-OFL-1.1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
Google Inc.

This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL


-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------

PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.

The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.

DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.

"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).

"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).

"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.

"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.

PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:

1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.

2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.

3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.

4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.

5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.

TERMINATION
This license becomes null and void if any of the above conditions are
not met.

DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
5 changes: 5 additions & 0 deletions assets/vendor/searchable-pdf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Searchable PDF assets

`fontkit.umd.min.js` is prepared from `@pdf-lib/fontkit` 1.1.1 under the MIT license. `NotoSansKR-400.js` contains the Noto Sans KR font bytes from `@fontsource/noto-sans-kr` 5.3.0 as base64 so the tool can load them through the existing same-origin `script-src` policy while retaining `connect-src 'none'`. Noto Sans KR is licensed under the SIL Open Font License 1.1. All assets are pinned, integrity checked, served from the same origin, and never fetched at runtime from a CDN.

The Korean font is embedded as a subset in generated PDFs. Updating either dependency requires running `npm run prepare:ocr` and reviewing the generated manifest.
9 changes: 9 additions & 0 deletions assets/vendor/searchable-pdf/fontkit-LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
MIT License

Copyright (c) 2021 Andrew Dillon

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
1 change: 1 addition & 0 deletions assets/vendor/searchable-pdf/fontkit.umd.min.js

Large diffs are not rendered by default.

21 changes: 21 additions & 0 deletions assets/vendor/searchable-pdf/manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"generatedBy": "npm run prepare:ocr",
"packages": {
"@pdf-lib/fontkit": "1.1.1",
"@fontsource/noto-sans-kr": "5.3.0"
},
"assets": {
"fontkit.umd.min.js": {
"bytes": 758440,
"sha256": "d8df561b9fba98e24f2e5130e40948809281bbbc55a20c412359f1a0a5eb35a6"
},
"NotoSansKR-OFL-1.1.txt": {
"bytes": 4314,
"sha256": "18aabf190848725e2576eefb5c29ba06aac1029d02132252a7f312eac2e50cf3"
},
"NotoSansKR-400.js": {
"bytes": 1133310,
"sha256": "04c744353dde85d9ee99ba690db122459628041be7735b14f72e62f05beb3f16"
}
}
}
8 changes: 4 additions & 4 deletions docs/pdf-ocr-foundation.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# PDF OCR foundation

The PDF OCR foundation is a shared, strict TypeScript pipeline for future PDF-to-text and Searchable PDF workflows. It is not currently exposed as a public tool route. Searchable PDF generation remains a separate Sprint.
The PDF OCR foundation is a shared, strict TypeScript pipeline used by PDF to Text and its Searchable PDF output. Layout-aware callers receive line text, confidence, raster dimensions, the inverse PDF.js viewport transform, and an existing-text signal from the same OCR pass.

## Pipeline

`tools/shared/pdf-ocr.ts` loads a local PDF with the existing same-origin PDF.js renderer, renders each selected page to a white-background PNG canvas, passes that page image to the existing Tesseract OCR service, and returns an ordered array of `{ pageNumber, text }` results. The pipeline preserves English, Korean, and English + Korean recognition only.
`tools/shared/pdf-ocr.ts` loads a local PDF with the existing same-origin PDF.js renderer, renders each selected page to a white-background PNG canvas, and passes that page image to the existing Tesseract OCR service. Text-only callers keep the ordered `{ pageNumber, text }` contract; layout-aware callers opt into line boxes and page geometry. The pipeline preserves English, Korean, and English + Korean recognition only.

Pages are processed sequentially with one OCR service. A render scale of `2` corresponds to approximately 144 pixels per PDF inch and balances OCR detail with browser memory. The existing 16,384-pixel dimension and 50-megapixel canvas safeguards remain in force. Documents with 25 or more pages surface a `largeDocument` signal rather than receiving an arbitrary hard rejection.

Expand All @@ -14,8 +14,8 @@ Progress reports document loading, page rendering, real OCR stages, completed pa

Cancellation destroys the active PDF renderer and forwards an abort signal to OCR. Generation checks prevent late progress and page results after cancellation or source replacement. Every page proxy is cleaned, every canvas is reset to 1 × 1, the renderer is destroyed after each job, and OCR disposal terminates its worker. No page image, text, or PDF is uploaded, and no object URL is required.

The core exposes review-ready page boundaries but no searchable PDF, positioned text layer, geometry mapping, or PDF output. A future public UI can add range parsing, review, copy, and TXT export around these typed contracts without changing the local processing boundary.
The PDF to Text route uses the layout-aware contract for both editable TXT output and Searchable PDF generation. This avoids a second OCR run while keeping older text-only callers compatible.

## Verification

Unit tests cover page selection, page numbering, all three OCR language contracts, progress, invalid input, retry, cancellation during rendering and recognition, source replacement, stale callbacks, large-document signaling, and resource cleanup. The internal browser harness performs real one-page PDF rendering and English OCR, real multi-page and larger-page rendering, cancellation timing, and same-origin request checks. Existing OCR smoke continues to execute real English, Korean, and combined recognition.
Unit tests cover page selection, page numbering, layout data, viewport inversion, all three OCR language contracts, progress, invalid input, retry, cancellation during rendering and recognition, source replacement, stale callbacks, large-document signaling, and resource cleanup. The internal browser harness performs real one-page PDF rendering and English OCR, real multi-page and larger-page rendering, cancellation timing, and same-origin request checks. Existing OCR smoke continues to execute real English, Korean, and combined recognition.
13 changes: 13 additions & 0 deletions docs/searchable-pdf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Searchable PDF output

Searchable PDF is integrated into `/pdf/to-text/` so page selection, English/Korean language choice, progress, cancellation, and the existing local OCR pass are shared. It adds no route, redirect, category entry, upload, telemetry, or network service.

The OCR service requests Tesseract line boxes and confidence values together with text. No confidence threshold is applied: low-confidence text is retained rather than silently discarded. PDF.js supplies the rendered viewport transform; one inverse affine transform maps raster coordinates back to PDF coordinates for 0°, 90°, 180°, 270°, CropBox offsets, and the fixed 2× render scale.

The writer loads and saves the original document with the repository's pdf-lib 1.17.1. It appends opacity-zero text using a subsetted local Noto Sans KR font, leaving original page content and page order in place. Pages with meaningful existing selectable text skip the added OCR layer, and pages outside the selected OCR set remain unchanged. The original visible page is never replaced with a raster image.

Noto Sans KR comes from `@fontsource/noto-sans-kr` 5.3.0 under the SIL Open Font License 1.1. The font and `@pdf-lib/fontkit` 1.1.1 runtime are pinned, integrity checked, and served from the same origin. There is no runtime CDN request.

pdf-lib preserves ordinary page content, page geometry, metadata, links, annotations, and forms while adding content streams. Complex PDFs can contain implementation-specific structures such as signatures, dynamic forms, optional content, or unusual outlines; the application does not promise bit-for-bit preservation or continued signature validity after modification. Password-protected PDFs remain unsupported.

Cancellation is checked before and between pages and before/after serialization. pdf-lib serialization itself is synchronous and cannot be interrupted mid-save; a generation guard prevents a cancelled or superseded result from downloading after serialization returns.
Loading
Loading