Build under the sanitizers and with -Werror in CI - #21
Merged
Conversation
Every pull request so far was checked under -fsanitize=address,undefined by hand and came back clean. That is the argument for making it a job rather than a habit: the habit lives in a scratch file outside the repo, and the next contributor does not have it. CI becomes a four-way matrix -- both compilers against an optimised and a sanitised build -- because -O2 and the sanitizers do not see the same program. The optimised build is what ships; the sanitised one is the only thing that reports a read past the end of a buffer rather than returning whatever was next to it. Three details that are the whole point: * -fno-sanitize-recover=undefined. UBSan's default is to print the diagnostic and carry on, exiting 0, so without this the entry is decorative: a real finding would be a green build, and nobody reads the log of a job that passed. Verified by injecting an out-of-bounds read into count_label(): the optimised entry sails past it, and the sanitised entry stops with exit 2 and names src/text.cpp and the test that reached it. * -Werror is here and not in the Makefile. A warning should stop a change being merged, not stop a contributor building the project at all, so `make` on a clean checkout stays warm. * CXX and CXXFLAGS are set for the job rather than passed to each make. The Makefile takes both with ?=, which honours the environment, and the test and golden targets have to link against objects built with the same flags -- passing them one step at a time is how you get a link error against a sanitizer runtime that half the objects do not have. UBSAN_OPTIONS=print_stacktrace=1 so that a failure names the source line rather than an address. All four entries pass locally on gcc 16 and clang 22. CI runs gcc 13 and clang 18, which warn about different things, so this may well surface something the newer compilers stopped complaining about -- which is the entry doing its job on its first run rather than a reason to hold it back.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every pull request so far was checked under
-fsanitize=address,undefinedby hand and came back clean. That is the argument for making it a job rather than a habit — the habit lives in a scratch file outside the repo, and the next contributor doesn't have it.CI becomes a four-way matrix: both compilers against an optimised and a sanitised build.
-O2and the sanitizers don't see the same program — the optimised build is what ships, and the sanitised one is the only thing that reports a read past the end of a buffer rather than returning whatever was next to it.Three details that are the whole point
-fno-sanitize-recover=undefined. UBSan's default is to print the diagnostic and carry on, exiting 0. Without this flag the entry is decorative: a real finding is a green build, and nobody reads the log of a job that passed. This is not theoretical — here it is both ways on a two-line program:-Werroris here and not in the Makefile. A warning should stop a change being merged, not stop a contributor building the project at all.makeon a clean checkout stays warm.CXX/CXXFLAGSare set for the job, not passed to eachmake. The Makefile takes both with?=, which honours the environment. Thetestandgoldentargets link against the objects built by theBuildstep, so they need identical flags — passing them one step at a time is how you get a link error against a sanitizer runtime that half the objects don't have.Plus
UBSAN_OPTIONS=print_stacktrace=1, so a failure names the source line rather than an address.Verifying the entry can actually fail
Green proves nothing on its own, so I injected an out-of-bounds read into
count_label()— a function the unit tests certainly reach — and ran two entries against it:It stopped rather than printing and passing, which is
-fno-sanitize-recoverdoing its job, and the stack trace names the test that reached it. The injected bug was reverted; the diff here isci.ymlplus docs.Testing
All four entries pass locally: 618 unit checks and 42 golden cases each.
g++clang++One thing worth flagging: locally that's gcc 16 and clang 22, while
ubuntu-latestis gcc 13 and clang 18. Older compilers warn about different things — notably the-Wmaybe-uninitializedand-Wdangling-referencefalse positives that later versions fixed. If this run surfaces something, that's the entry doing its job on its first outing rather than a reason to hold it back, and I'll fix what it finds.