Redirect 34 orphaned pages that the deploy never removed - #437
Open
eugenia-scandit wants to merge 1 commit into
Open
Redirect 34 orphaned pages that the deploy never removed#437eugenia-scandit wants to merge 1 commit into
eugenia-scandit wants to merge 1 commit into
Conversation
/sdks/*/samples/ (10 pages) and /sdks/*/id-validate/* (24 pages) are still served from docs.scandit.com but exist nowhere in docs/ and in no sitemap. The S3 deploy does not delete files the build stops producing, so every one of them still serves its old build's HTML — including its old docusaurus_tag. Provenance, so the dates stay checkable (`git log --diff-filter=D` over the deleted paths): - samples: removed 2025-12-11 in 43e447d (#277), all 10 frameworks including titanium. - id-validate: removed across three Nov 2024 commits — 3893cfc (11-14) android, ios; edc7107 (11-18) capacitor, cordova, web; 39011c4 (11-21) flutter, net/android, net/ios, react-native and xamarin/{android,forms,ios}. Two consequences, and the second is the reason this matters. The site's own search filters on docusaurus_tag, so 527 records under docs-default-8.0.0 and docs-default-6.28.1 are unreachable from the search box — counted by browsing the Algolia index and grouping by docusaurus_tag. But the PAGES are fully reachable by URL, by Google and by AI crawlers, serving documentation up to two years old with nothing marking it stale. Note that this change does not fix those 527 search records: a redirect replaces HTML on S3, it does not delete an index record. Reindexing those tags is a separate pass. A redirect emits a file at each path, so the next deploy OVERWRITES the orphan. That is why this is a redirect and not a crawler exclusion: an exclusion would only hide the pages from our own search while Google kept serving them. (That key is `exclusionPatterns`, and it belongs to the Algolia Crawler config, which lives outside this repository.) Targets follow what the docs already do: - ID Validate folded into ID Capture from v7 on, and ID Capture has intro and get-started for all 9 frameworks, so this maps page-for-page instead of dumping readers on a hub. The v6 content was not lost: it is still built and still sitemapped at /6.28.11/sdks/<fw>/id-validate/*. - Titanium is deliberately absent from the id-validate list — it still has real current id-validate pages, and listing it would make the build throw. It IS in the samples list, because 43e447d removed its samples.md. That is why one list has 9 entries and the other 10; do not "complete" either. - Samples moved to GitHub, reusing the destination already used by the /data-capture-sdk/<fw>/samples/run-samples.html entries further down the array. net/ios and net/android share the dotnet repo. - The 6 xamarin id-validate pages go through the xamarin `commonPages` list in createRedirects, not the `redirects` array, so they inherit the existing /migrate-7-to-8 target like every other xamarin page. Verified: build exits 0, emits all 34 stubs (18 id-validate + 10 samples + 6 xamarin), no redirect or duplicate-route warnings, and the real pages survive — web/id-capture/intro, linux/samples, titanium/id-validate/intro and 6.28.11/sdks/xamarin/android/id-validate/intro are all still built as pages. THIS DOES NOT FIX THE CAUSE, AND IT ONLY REACHES PART OF THE PROBLEM. The 34 root-level pages here are the covered case. The larger case is the retired VERSION prefixes: 21 trees are still live — /7.6.3/ through /7.6.13/ and /6.28.1/ through /6.28.10/, roughly 450-500 pages each, so on the order of 10,000 orphaned pages against the 34 fixed here. (/8.5.3/, /8.5.2/, /8.4.1/ and /8.3.1/ return 404, so those were cleaned up.) Redirects can never reach them: the build emits nothing under /7.6.3/, so no redirect file can ever land there. Fixing the deploy is therefore not optional cleanup — it is the only thing that addresses the bulk. That fix is a SCOPED PRUNE, not a bare `--delete`. An earlier version of this commit message recommended `aws s3 sync --delete` as "the real fix"; that recommendation was wrong and would have caused an outage. An unqualified `--delete` at the bucket root deletes every path this build does not own, and the API reference is exactly that — a separate Sphinx build published to the same origin by a different pipeline. Verified against this build's output: it emits only 78 redirect stubs under data-capture-sdk/ and nothing at all under 7.6/ or 6.28/, so a root `--delete` would take the entire API reference pages) off the live site. What is needed is a prune limited to the prefixes this build owns — a `--delete` scoped by `--exclude` for `data-capture-sdk/*`, `7.6/*`, `6.28/*` and the retired version prefixes, or a manifest diff of build output against the bucket. The origin layout must be confirmed with whoever owns the deploy pipeline before any form of this is run: the exclusion list above is derived from one local build, not from the bucket. That pipeline lives outside this repository. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/sdks//samples/ (10 pages) and /sdks//id-validate/* (24 pages) are still served from docs.scandit.com but exist nowhere in docs/ and in no sitemap. The S3 deploy does not delete files the build stops producing, so every one of them still serves its old build's HTML — including its old docusaurus_tag.
Provenance, so the dates stay checkable (
git log --diff-filter=Dover the deleted paths):Two consequences, and the second is the reason this matters. The site's own search filters on docusaurus_tag, so 527 records under docs-default-8.0.0 and docs-default-6.28.1 are unreachable from the search box — counted by browsing the Algolia index and grouping by docusaurus_tag. But the PAGES are fully reachable by URL, by Google and by AI crawlers, serving documentation up to two years old with nothing marking it stale. Note that this change does not fix those 527 search records: a redirect replaces HTML on S3, it does not delete an index record. Reindexing those tags is a separate pass.
A redirect emits a file at each path, so the next deploy OVERWRITES the orphan. That is why this is a redirect and not a crawler exclusion: an exclusion would only hide the pages from our own search while Google kept serving them. (That key is
exclusionPatterns, and it belongs to the Algolia Crawler config, which lives outside this repository.)Targets follow what the docs already do:
commonPageslist in createRedirects, not theredirectsarray, so they inherit the existing /migrate-7-to-8 target like every other xamarin page.Verified: build exits 0, emits all 34 stubs (18 id-validate + 10 samples + 6 xamarin), no redirect or duplicate-route warnings, and the real pages survive — web/id-capture/intro, linux/samples, titanium/id-validate/intro and 6.28.11/sdks/xamarin/android/id-validate/intro are all still built as pages.