test: use a non-credential-shaped sentinel in the leak tests - #2
Merged
dmccoystephenson merged 1 commit intoSep 1, 2026
Merged
Conversation
The leak tests hand the client a stand-in secret and assert it never reaches a returned value, a URL or an error message. The stand-in was literally `ghp_...`-prefixed. That was fine while the repo was private; now that it is public, GitHub secret scanning matches the literal, files a false-positive alert, and push protection can block any push that touches the line. The alert is noise, but recurring noise is worse than no alert: it trains people to dismiss the one that is real. The tests are unchanged in what they check. They only ever needed a distinctive, greppable sentinel to assert the absence of, and `sentinel-not-a-real-credential-leak-canary` does that job identically while matching no scanner pattern and carrying no entropy. Both files carry a comment explaining why the fixture deliberately does not look authentic, so it does not get "corrected" back later. Verified by removing the redaction in the Python client: the leak test fails on the sentinel exactly as it did on the old value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
js/test/client.test.tsandpython/tests/test_client.pyboth hand the client a stand-in secret and assert it never reaches a returned value, a request URL, or an error message. Good tests — but the stand-in was a literalghp_-prefixed string.That was harmless while this repo was private. Now that it is public, GitHub secret scanning matches the literal, files a false-positive alert, and push protection can block any future push that touches those lines.
The alert is noise. The problem with recurring noise is that it trains people to dismiss the alert that is real.
What changed
The fixture value only, in both suites:
Both files gained a comment explaining that the value deliberately avoids the
ghp_/gho_/github_pat_prefixes and any random-looking entropy, so nobody later "fixes" it back into something authentic-looking.What did not change
The assertions. The tests never needed the fixture to look like a credential — they needed a distinctive, greppable sentinel to assert the absence of, and the replacement is exactly as good at that. Every call site (
Authorization: Bearer ${TOKEN}, the leaky-error-body construction, the URL check, thenot.toContain/assertNotInassertions) is untouched.Verification
Both suites pass, and the leak assertion was mutation-checked: with
GitHubDocsClient._redactstubbed out to return its input unchanged,test_an_upstream_error_that_echoes_the_token_is_redactedfails on the new sentinel exactly as it would have on the old value. The redaction was restored before committing.