Skip to content

build: build the firmware with CMake - #1967

Merged
gmarull merged 17 commits into
coredevices:mainfrom
teslabs:build/cmake
Aug 31, 2026
Merged

build: build the firmware with CMake#1967
gmarull merged 17 commits into
coredevices:mainfrom
teslabs:build/cmake

Conversation

@gmarull

@gmarull gmarull commented Aug 30, 2026

Copy link
Copy Markdown
Member

Replaces the waf firmware build with CMake, driven by Ninja.

./pbl configure --board asterix
./pbl build

What changes

The build model is the one waf had grown into, which was itself borrowed
from Zephyr, so the shape of a directory's build file is unchanged —
declare a library, give it sources, gate the optional pieces on Kconfig:

pbl_library()
pbl_library_sources(service.c)
pbl_library_sources_ifdef(CONFIG_MAG correction.c)

What the build does is unchanged too. Kconfig still resolves the board's
configuration into .config and autoconf.h; the resource pipeline, the
generated tables and headers, the linker script hooks, the C library
selection and the image and bundling steps all keep their behaviour.

The generators the build shells out to move from waf task classes to plain
command line tools under tools/cmake/, and the helpers that were never
waf-specific (board parsing, the pblboot header, SLE) move out of
tools/waf/ next to the rest of the tooling.

waf stays for what it still builds: the unit tests, out of their own build
directory, and the SDK packaging (./pbl waf sdk).

docs/development/build_system.md is rewritten around the new API.

Verification

  • qemu_flint: byte-identical system resources, and an ELF with the same
    symbols at the same sizes, differing only in padding inside .text.
  • asterix: identical .text / .bss / .log_strings sizes and an
    identical symbol table.
  • Firmware, PRF, QEMU and SDK-shell workflows all build in CI.

Why CMake and not Meson

A full Meson port was written and benchmarked against this branch before
settling. Same board, same machine, ccache off:

CMake Meson
configure 1.4 s 4.3 s
clean build 48.8 s 49.7 s
null build 0.30 s 0.31 s
rebuild one file 1.9 s 3.3 s
reconfigure 1.0 s 3.9 s

Both emit Ninja, so the build itself is a wash. The gaps are all
configure-time, and the deciding factor was expressiveness: Meson has no
user-defined functions, so pbl_library() becomes a dict literal the root
assembles later, which added ~950 lines to the per-directory files
contributors actually touch. It also has no per-source compile flags
(__FILE_NAME_LEGACY__ needed a compiler wrapper, which rules out
ccache), no globbing and no CONFIGURE_DEPENDS, cannot read a file out of
the build directory, cannot name a subdirectory in a custom_target
output, and has one flat variable scope shared across subdir().

The steps that turn resources into the firmware's tables, headers and
pbpack were written as waf task classes, so nothing else could call them.
Move the logic into tools/resources/generators.py, as plain functions over
paths, and leave the task classes as the thin wrappers they should have
been.

The SDK build, which keeps using waf, gets the new module bundled
alongside the waftools it already ships.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
@gmarull

gmarull commented Aug 30, 2026

Copy link
Copy Markdown
Member Author

Build performance, measured against waf

I profiled the build and pushed three commits on top of the migration. All
numbers below are asterix, ccache off, on the same 8-core machine in one
sitting, so the three columns are comparable to each other (the machine was
busier than when the numbers in the PR description were taken, so the
absolute values are higher — read the columns against each other, not
against the description).

waf (main) CMake (before) CMake (now)
configure 1.5 s 2.7 s 2.3 s
clean build 99.1 s 94.9 s 68.3 s
no-op build 2.6 s 0.7 s 0.5 s
rebuild one file 3.3 s 3.8 s 1.6 s
reconfigure 0.8 s 1.7 s 1.4 s

Against waf that is 31 % off a clean build, 5× off a no-op and 2× off the
inner loop; configure is the one step that is slower, by about 0.8 s, which
is CMake's compiler detection and find_package(Python3).

The firmware is unchanged by all three commits: same pebbleos.bin, same
loadable sections, same symbols at the same addresses. The ELF drops from
24 MB to 17 MB and the ninja graph from 2892 to 2351 edges.

Where the time was going

Clean build: 63 % compiling firmware, 18 % generating resources, 15 %
compiling third-party code, the rest linking and archiving. Inside a
compile, 57 % is the preprocessor — and that is header content, not the
search path: passing only the include directories a file actually uses
(a median of 7 of the 96 it is given) makes no measurable difference, and
-O0 is not faster than -Os. No-op build: 78 % of it was one rule.

What the three commits do

  1. build: generate resources in batches — every one of the 581
    resources was its own Python process, and 77 ms of the ~100 ms each took
    was the interpreter and its imports. 120 resources cost 9.8 s of CPU one
    process at a time and 0.5 s in a single process. They are now grouped by
    type and built 16 at a time: 40 rules instead of 581, ~45 s of CPU off a
    clean build.
  2. build: make the macro debug info and the linker map opt-in-g3
    costs ~7 % of the compile and a 3.4 MB .debug_macro section that also
    slows the link; -Wl,-Map with --cref writes tens of megabytes and is
    about a fifth of the link. Both are Kconfig symbols now
    (CONFIG_DEBUG_INFO_MACROS, CONFIG_LINKER_MAP), off by default. Turn
    the second one on before running
    tools/analyze_fw_static_memory_usage.py.
  3. tools: skip submodules in the firmware version's dirty check — the
    version header is regenerated on every build, and git describe --dirty
    was 0.24 s of the 0.31 s that took, all of it walking into the
    submodules. The dirty flag is now a separate check that ignores their
    working trees (a submodule moved to another commit still counts).

Left on the table

The remaining 63 % is header fan-out — 57 % of every compile is the
preprocessor working through PebbleOS's headers. That is a source refactor
rather than a build-system change, so it is not in this PR, but a histogram
over ninja -t deps would name the worst offenders. Splitting the global
include path per library would also shrink build.ninja, 75 % of which is
-I flags, though it buys no compile time.

@gmarull
gmarull force-pushed the build/cmake branch 4 times, most recently from 92a7457 to d2f8a2c Compare August 30, 2026 17:53
@gmarull
gmarull marked this pull request as ready for review August 31, 2026 09:10
@gmarull
gmarull requested a review from jplexer as a code owner August 31, 2026 09:10
gmarull and others added 16 commits August 31, 2026 12:51
Replace the waf firmware build with CMake, driven by Ninja. The model
is the one waf had grown into, which was itself borrowed from Zephyr, so
the shape of a directory's build file is unchanged: declare a library,
give it sources, gate the optional pieces on Kconfig symbols.

    pbl_library()
    pbl_library_sources(service.c)
    pbl_library_sources_ifdef(CONFIG_MAG correction.c)

What the build does is unchanged too. Kconfig still resolves the board's
configuration into .config and autoconf.h; the resource pipeline, the
generated tables and headers, the linker script hooks, the C library
selection and the image and bundling steps all keep their behaviour, and
the firmware they produce is the same: for qemu_flint, byte-identical
system resources, and an ELF with the same symbols at the same sizes,
differing only in padding inside .text.

The generators the build shells out to move from waf task classes to
plain command line tools under tools/cmake/, and the helpers that were
never waf-specific (board parsing, the pblboot header, SLE) move out of
tools/waf/ next to the rest of the tooling.

waf stays for what it still builds: the unit tests, out of their own
build directory, and the SDK packaging (`./pbl waf sdk`).

The object file names change with the build system, and the sf32lb52x
ramfunc fragment selects the code that has to run from RAM by matching
them: waf wrote bf0_hal_mpi.c.1.o, CMake writes bf0_hal_mpi.c.obj. Its
patterns end wild so they match either.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The firmware workflows call ./pbl, which now drives CMake; the test
workflow keeps waf and follows the unit tests into build-test/.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Rewrite the build system page around CMake and the pbl_library API, point
the testing and SDK pages at their new build directory and commands, and
add CMake to the nix development shell.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Every one of the board's several hundred resources was a build rule of
its own, and each rule was a Python process that spent more time starting
up than generating. 120 resources cost 9.8 s of CPU one process at a
time and 0.5 s in a single process; the interpreter and its imports are
77 ms of the roughly 100 ms a resource takes.

Group them by type -- so a process still imports only the one generator
module it needs -- and build them sixteen at a time. That is 40 rules
instead of 581, small enough that they still spread across the cores, and
takes about 45 s of CPU out of a clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Two build outputs nobody asks for most of the time. -g3 keeps every macro
definition in the debug info, which costs about 7% of the compile and
grew the ELF from 17 to 24 MB, most of it a 3.4 MB .debug_macro section
that also slows the link down. -Wl,-Map with --cref writes tens of
megabytes and is about a fifth of the link; the only thing that reads it
is tools/analyze_fw_static_memory_usage.py.

Both are now Kconfig symbols, off by default. The firmware itself does
not change: same loadable sections, same symbols at the same addresses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The version header is regenerated on every build, so that a new commit
always reaches the firmware, and `git describe --dirty` was 0.24 s of the
0.31 s that took. All of it is the walk into the submodules: the same
check that ignores their working trees is 0.02 s.

Ask git for the tag and the dirty flag separately, and let the flag
ignore uncommitted edits inside a submodule -- a submodule moved to
another commit is still a change, and still counts. Combining the commit
and timestamp lookups into one process saves another call. A no-op build
goes from 0.72 s to 0.46 s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The firmware needs CMake and Ninja, which the v6 image does not carry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
They were built from execute_process() while CMake was still configuring,
which is not where a compile belongs: it needs a host compiler, has
nothing to do with resolving the configuration, and any failure aborts
the configure rather than the build. waf ran them as a build task; this
puts them back there, as the custom command that produces mc.xs.c and
mc.resources.c.

Configuring qemu_emery with CONFIG_SHELL_SDK=y drops from minutes to 1.6 s
as a result, since the host tools are no longer built to answer a
question nobody asked.

Moving them also means the headers mcconfig writes are no longer there
before the first compile, so the generated set -- mc.defines.h and the
rest, which xsHost.h pulls in -- hangs off pbl_generated_headers, which
every library already waits for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
set(ENV{PATH}) only reaches what CMake runs while it is configuring. The
generators shell out to the toolchain by name -- the native SDK asks for
arm-none-eabi-gcc, the stored apps' metadata injection for
arm-none-eabi-nm -- so on a machine where the toolchain is not already on
PATH, which is every CI runner, they failed as soon as they ran as build
steps rather than configure ones.

Every generator now runs behind `cmake -E env PATH=...`, with the PATH the
SDK locator resolved at configure time. The steps that still run while
configuring are left alone: they already inherit it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
A Pebble app is a single blob: the firmware loads it into app RAM and
runs it there, so pebble_app.ld gives it one rwx region and everything
lands in one LOAD segment. binutils 2.39 warns about that.

Splitting the segment would be worse than the warning -- ld pads between
LOAD segments, and the flat image objcopy produces is what the firmware
loads -- so tell ld the permissions are deliberate. The option is only
passed when the linker knows it, and the app binary is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
A <path> whose d is only a move draws nothing, and the icon set has 104
of them across 49 files -- stray anchor points the drawing tools leave
behind. Skipping them loses no ink, so the generator printed a hundred
lines of "No points in parsed path" per build, with no file named and
nothing to act on.

Say it only under --verbose, and say which path it was. The diagnostics
that do mean dropped artwork -- unsupported elements, unrecognized
circles -- still print unconditionally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
check-size exists to fail the build when the image no longer fits; the
running total it printed on every build was noise. The size limit is
still enforced, and still says how far over it is when it trips.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
FW_FLASH_SIZE was the whole 1 MiB, but the image is linked from
FLASH_OFFSET, 32 KiB in, past the bootloader -- so the FLASH region ran
to 0x108000, past the end of the part, and the linker would accept an
image 32 KiB larger than fits.

Every other board already subtracts what it reserves. The link now
reports 992 KiB, and the region is the one the image is flashed into.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The linker enforces the same limit from the FLASH region and fails the
link with the overflow, so checking the size of the image afterwards only
repeated the answer -- and printed it on every build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The last copy of the limits, and the same reasoning: the linker will not
produce an image that does not fit its FLASH region, so re-measuring the
binary before flashing it answered a question already settled. The
exception it raised was never caught either, so an oversize image would
have come out as a traceback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Interrupting a build unwound through the subprocess wait() it landed in
and printed a traceback of Python internals, which says nothing about
what was interrupted.

Ctrl+C reaches the whole process group, so ninja has already stopped and
said so; report 128 + SIGINT, as a shell expects from an interrupted
command, and let its message be the last word.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
@gmarull
gmarull merged commit 826cad3 into coredevices:main Aug 31, 2026
49 checks passed
@gmarull
gmarull deleted the build/cmake branch August 31, 2026 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant