Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 0 additions & 20 deletions .gcovr.cfg

This file was deleted.

21 changes: 20 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,25 @@ on:
workflow_dispatch:

jobs:
# A tag push publishes whatever version pyproject.toml declares, so a mistyped
# tag would ship the wrong version silently. Fail fast instead.
check_version:
name: Tag matches project version
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
steps:
- uses: actions/checkout@v4
- name: Compare tag with pyproject version
run: |
python -c "
import tomllib, os, sys
version = tomllib.load(open('pyproject.toml','rb'))['project']['version']
tag = os.environ['GITHUB_REF_NAME'].removeprefix('v')
if tag != version:
sys.exit(f'tag {tag!r} != pyproject version {version!r}')
print(f'tag matches project version: {version}')
"

build_wheels:
name: Wheel (${{ matrix.os }}, ${{ matrix.cibw_archs }})
runs-on: ${{ matrix.os }}
Expand Down Expand Up @@ -95,7 +114,7 @@ jobs:

publish_pypi:
name: Publish to PyPI
needs: [build_wheels, build_sdist, test_wheels_centos_stream_9]
needs: [check_version, build_wheels, build_sdist, test_wheels_centos_stream_9]
runs-on: ubuntu-latest
environment: pypi
permissions:
Expand Down
43 changes: 43 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,49 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.8.0] - 2026-08-24

### Added

- **Flow Policy**: `set_static_paths(src, dst, paths)` pins a demand to explicit path bundles (MPLS-style routing), finishing a feature that was ported but never bound and whose port bound every flow to the first bundle. One flow per usable bundle, each bound to its own bundle in supply order; bundles are validated against the graph and pruned against the policy's masks, and a bundle with no surviving `src->dst` walk is down (no reroute). `EqualBalanced` spreads over the up bundles only. With static paths the policy neither grows its flow set nor reoptimizes; `max_path_cost`/`-factor`, `min_flow_count` and `reoptimize_flows_on_each_placement` are inert.
- **Shortest Paths**: `PredDAG.from_edges(graph, edges)` builds a single-path `PredDAG` from a contiguous edge-id sequence (`make_path_dag` in C++) -- the missing constructor for operator-defined explicit paths. Extracting it also deduplicates the two identical path-to-DAG conversion blocks in `k_shortest_paths` (outputs unchanged).

### Fixed

- **Max-Flow**: `calc_max_flow` could return less than the true maximum, since the tier loop augments only along forward SPF DAGs and never cancels an earlier placement (the reported `min_cut` then contradicted `total_flow`). Added a residual completion phase with reverse arcs. **This is the default configuration** of `max_flow`/`batch_max_flow` (`Proportional` + `require_capacity=True` + `shortest_path=False`), so recorded expected values, regression baselines and stored results must be regenerated: `total_flow` increases where the previous value was suboptimal, and `min_cut`, `edge_flows`, `residual_capacity` and `reachable_nodes` change with it. `sensitivity_analysis`, which is built on `calc_max_flow`, is affected too.
- **Shortest Paths**: Zero-cost edges produced a cyclic predecessor DAG, making `EqualBalanced` placement return 0 flow and `resolve_to_paths` hang. Equal-cost predecessors are now recorded only while a node is unsettled, and `resolve_to_paths` guards against cycles in caller-supplied DAGs. Side effect on graphs containing zero-cost edges: among nodes mutually reachable at equal distance the settle order now decides which alternatives are kept, so a `PredDAG` can hold fewer equal-cost predecessors than in 0.7.2 and ECMP fan-out narrows accordingly. Graphs with strictly positive costs are unaffected (verified bit-identical).
- **K-Shortest Paths**: Spur enumeration materialized every equal-cost path, exponential in ECMP fan-out (a 67-node ECMP ladder needed ~4 s and ~3.5 GiB for `k=3`, and a 73-node one ~19 s; both are now instant in ~34 MB). Enumeration is now bounded by the number of candidates still acceptable; tie-breaking among equal-cost paths may differ, but path counts and costs are unchanged.
- **Flow Policy**: `place_demand` silently routed a second `(src, dst)` pair over the first pair's paths, and neither `place_demand` nor `rebalance_demand` checked that the supplied `FlowGraph` wrapped the policy's own graph -- with equal edge counts that selected paths on one topology and placed flow on another. Both now raise `invalid_argument` (`ValueError` from Python); use one policy per demand, or call `remove_demand()` first to retarget it.
- **Graph Construction**: `from_arrays` now rejects a total edge cost at or above 2^62, which overflows the int64 path arithmetic in SPF and silently corrupts results. The bound is on the sum of all edge costs -- a conservative upper bound on any path -- so it can reject a graph whose individual paths could not actually overflow.
- **Flow State**: `compute_min_cut` and max-flow reachability derived placed flow from `capacity - residual`, overstating it for a `FlowState` built with a custom `residual_init`. Both now use `edge_flow`.
- **K-Shortest Paths**: `k_shortest_paths` read `paths.back()` on an empty vector when `max_cost_factor < 1.0` put the cost ceiling below the shortest path, crashing for `k > 1` (the `k == 1` early return masked it). All `k` now return no paths for such a factor.
- **Python Bindings**: `batch_max_flow` rejected a valid int32 `pairs` array on Windows, because the dtype check compared buffer format strings and NumPy spells int32 as `NPY_LONG` on LLP64 but `NPY_INT` on LP64. It now compares dtype equivalence, and rejects non-contiguous `pairs` rather than reading them as if packed.
- **Build**: `NETGRAPH_CORE_SANITIZE` passed its flags as one quoted string, so sanitizer builds never compiled, and the test target was missing them entirely. `make sanitize-test` also no longer sets `detect_leaks=1` on macOS, where it aborts.
- **Build**: Coverage builds now use `-fprofile-update=atomic` on GCC; the default non-atomic counters corrupt under the threads used by `batch_max_flow` and `sensitivity_analysis`.

### Changed

- **Max-Flow**: `FlowSummary.costs`/`flows` is now a cost-weighted breakdown of the total flow rather than a list of path costs. Entries added by the new completion phase are *marginal* costs -- the augmenting path's forward edge costs minus the cost of the flow it cancels -- so an entry need not correspond to any traversable `src->dst` path. Consumers that treat these as path costs (for example a cost-distribution report) should be reviewed.
- **Flow Policy**: `max_path_cost` no longer implies a relative factor of 1.0. Previously `max_path_cost_factor` defaulted to 1.0 whenever either limit was set, so setting `max_path_cost` alone also rejected every path costing more than the best path found -- making the absolute limit unreachable. The two limits are now independent, and a policy with only `max_path_cost` set admits every path up to that cost. (Found while fixing undefined behavior in the same expression: with an unreachable destination, `best_path_cost_` is the `INT64_MAX` sentinel and multiplying it by the factor overflowed the cast back to `int64`.)
- **Shortest Paths**: Replaced nested per-node predecessor vectors with flat intrusive lists, removing the allocation churn that dominated the hot path (~3-4x faster; output unchanged).
- **Flow Placement**: `place_on_dag` now rebuilds its `(parent, child)` edge groups into a reused arena instead of nested per-node vectors, which was ~70% of its runtime (~2-2.4x faster; output unchanged).
- **Max-Flow**: Parallelized `batch_max_flow` across source/destination pairs using `std::async`; workers claim pairs from a shared counter, so a batch whose costs are unevenly distributed still parallelizes. Thread count from `NGRAPH_CORE_BATCH_THREADS` env or hardware concurrency; set it to `1` when calling from your own worker pool to avoid oversubscription.
- **BREAKING** **C++ API**: `FlowPolicy::set_static_paths` changed signature from `(std::vector<std::tuple<NodeId, NodeId, PredDAG, Cost>>)` to `(NodeId src, NodeId dst, std::vector<PredDAG> bundles)`, and its behavior changed: the previous implementation handed every flow the first matching bundle instead of one bundle per flow. It was never bound to Python, so only C++ callers are affected.
- **Flow Policy**: `place_demand` computes one SPF when seeding initial flows instead of repeating an identical one per flow; seeding cost no longer scales with `min_flow_count`.
- **Flow Policy**: The `EqualBalanced` rebalance recursion (`place_demand` -> `rebalance_demand` -> `place_demand`) is now an iterative loop with numerically identical results (returned leftover may differ by ~1 ulp from floating-point summation order); with many pinned bundles of heterogeneous capacity the recursion depth grew like `U * ln(imbalance/kMinFlow)`, a stack-overflow risk on worker threads.
- **Python Bindings**: Three input errors now raise where they previously did not, or raise a different type -- adjust `except` clauses accordingly. `Algorithms.spf` raises `TypeError` (was `ValueError`) for a wrong `residual` length, matching every other length check; `batch_max_flow` raises `ValueError` for out-of-range node ids that previously produced a zero-flow summary absorbed silently into the batch; and `Algorithms.ksp` validates `dtype` up front, where an invalid value previously went unnoticed whenever the query returned no paths.
- **Python Bindings**: A wrong-typed `graph`/`algorithms` argument now raises `TypeError` instead of an opaque `RuntimeError: Unable to cast ... to C++ type '?'`.
- **Python Typing**: Corrected `_docs.py` (pybind11 enums and classes are not `enum.Enum`/`dataclass`, `MinCut.edges` is an `int32` array, removed a `Path` class that never existed, and added the constructors for `FlowIndex`, `FlowPolicyConfig` and the three enums) and widened the type-checking re-exports from 6 names to all 16, so `Algorithms`, `FlowPolicy`, `StrictMultiDiGraph` and friends are no longer `Unknown` to type checkers despite the shipped `py.typed`. A missing stub constructor is now worse than no stub, so a test asserts every stub constructor matches its binding.
- **Tests**: Several assertions could never fail and now do: a self-loop assertion sat inside `except Exception: pass`, a thread-safety test ended in `assert True` while its workers called `pytest.fail()` from a thread where it cannot fail the test, and two tautologies (`assert size >= 0`) were replaced with the properties actually under test.
- **Docs**: Corrected README/CONTRIBUTING (required CMake is 3.23 not 3.15, `make py-test` does not exist, `make test` does not collect coverage), the `sensitivity_analysis` description in README and `backend.hpp` (it measures flow *lost* on edge removal, not gain from relaxing capacity), the zero-copy and GIL claims, and header contracts for `from_arrays`, `PredDAG`, `FlowSummary.costs`, `calc_max_flow` and `FlowPolicy`.

### Removed

- **Dead code**: `.gcovr.cfg` (never read -- gcovr's default config name has no leading dot, and `make cov` passes every flag explicitly), the unreachable `FlowGraph` include and `expect_flow_conservation()` helper in the C++ test utilities, unused test helpers/fixtures, `_USE_MATH_DEFINES` (no `M_*` macro is used), and unused `<optional>`/`<stack>`/`<unordered_map>`/`<unordered_set>`/`<cmath>` includes across `src/` and `include/`.
- **BREAKING** **Python Bindings**: `Algorithms.build_graph_from_arrays`. Nothing used it (no test, and not NetGraph), and the `Graph` handle it returned could not be passed to `FlowGraph` or `FlowPolicy`, so it only ever served the stateless algorithms while looking like a peer of `build_graph`. Build the graph with `StrictMultiDiGraph.from_arrays()` and pass it to `Algorithms.build_graph()`.
- **BREAKING** **C++ API**: The `build_graph(std::shared_ptr<const StrictMultiDiGraph>)` overload on `Algorithms` and `Backend`, which existed solely to serve that binding. It was a pure virtual, so an out-of-tree `Backend` implementation must delete its now-dangling `override` to compile. Callers wanting the handle to own the graph construct it directly: `GraphHandle{my_shared_ptr}`.
- **BREAKING** **Python Bindings**: The `Flow` class (bound from `FlowRecord`). It was reachable only as `_netgraph_core.Flow`, never exported from `netgraph_core`, and no binding constructed or returned one, so nothing could obtain an instance; the name also collided with the C++ alias `using Flow = double`.

## [0.7.2] - 2026-03-26

### Fixed
Expand Down
26 changes: 22 additions & 4 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,11 @@ add_executable(netgraph_core_tests
target_link_libraries(netgraph_core_tests PRIVATE netgraph_core GTest::gtest_main)
target_include_directories(netgraph_core_tests PRIVATE tests/cpp)
if(NETGRAPH_CORE_COVERAGE AND (CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang"))
# Atomic counters: see the NETGRAPH_CORE_COVERAGE block below.
target_compile_options(netgraph_core_tests PRIVATE -O0 -g --coverage)
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
target_compile_options(netgraph_core_tests PRIVATE -fprofile-update=atomic)
endif()
target_link_options(netgraph_core_tests PRIVATE --coverage)
endif()
include(GoogleTest)
Expand Down Expand Up @@ -85,7 +89,6 @@ target_include_directories(netgraph_core PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>
)
target_compile_definitions(netgraph_core PRIVATE _USE_MATH_DEFINES)
## C++ standard is set globally above.
target_compile_features(netgraph_core PUBLIC cxx_std_20)

Expand Down Expand Up @@ -206,8 +209,15 @@ endif()
if(NETGRAPH_CORE_COVERAGE)
message(STATUS "Enabling coverage instrumentation (disables optimizations)")
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
# batch_max_flow and sensitivity_analysis run worker threads, and the default
# non-atomic profile counters corrupt under concurrent updates (gcov then
# emits impossible values such as "branch 4 taken -1").
set(COV_FLAGS -O0 -g --coverage)
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
list(APPEND COV_FLAGS -fprofile-update=atomic)
endif()
foreach(tgt netgraph_core _netgraph_core)
target_compile_options(${tgt} PRIVATE -O0 -g --coverage)
target_compile_options(${tgt} PRIVATE ${COV_FLAGS})
target_link_options(${tgt} PRIVATE --coverage)
endforeach()
else()
Expand All @@ -218,8 +228,16 @@ endif()
if(NETGRAPH_CORE_SANITIZE)
message(STATUS "Enabling sanitizers (disables optimizations)")
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
set(SAN_FLAGS "-fsanitize=address,undefined -fno-omit-frame-pointer")
foreach(tgt netgraph_core _netgraph_core)
# Must be a CMake list: a single quoted string reaches the compiler as one
# argument and clang rejects "-fsanitize=address,undefined -fno-omit-frame-pointer".
set(SAN_FLAGS -fsanitize=address,undefined -fno-omit-frame-pointer)
set(SAN_TARGETS netgraph_core _netgraph_core)
if(TARGET netgraph_core_tests)
# The test executable links the sanitized static library, so it must be
# built and linked with the same flags to pull in the sanitizer runtime.
list(APPEND SAN_TARGETS netgraph_core_tests)
endif()
foreach(tgt ${SAN_TARGETS})
target_compile_options(${tgt} PRIVATE ${SAN_FLAGS})
target_link_options(${tgt} PRIVATE ${SAN_FLAGS})
endforeach()
Expand Down
20 changes: 13 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Run specific test suites:

```bash
make cpp-test # C++ tests (GoogleTest)
make py-test # Python tests (pytest)
make test # Python tests (pytest)
```

### Code Style
Expand All @@ -65,13 +65,19 @@ make format

Releases are automated via GitHub Actions when a new tag is pushed.

1. Bump version in `pyproject.toml`.
2. Commit and push.
3. Create and push a tag:
1. Move the pending notes into a new `## [X.Y.Z] - YYYY-MM-DD` section in `CHANGELOG.md`,
dated the day you tag. Mark anything that breaks consumers as **BREAKING**.
2. Bump `version` in `pyproject.toml` to the same `X.Y.Z`. CI does not cross-check the
tag against this value, so a mismatch publishes the wrong version silently.
3. Commit and push.
4. Create and push a matching tag:

```bash
git tag v0.1.0
git push origin v0.1.0
git tag v0.8.0
git push origin v0.8.0
```

4. The CI pipeline will build wheels, sdist, and publish to PyPI.
5. The CI pipeline builds wheels and the sdist, tests them, and publishes to PyPI via
Trusted Publishing. `make publish` / `make publish-test` upload whatever is in your
local `dist/` and are for Test PyPI or emergencies only -- real releases go through
the tag.
6 changes: 4 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ help:
@echo " make check-ci - Run non-mutating lint + tests (CI entrypoint)"
@echo " make lint - Run only linting (non-mutating: ruff + pyright)"
@echo " make format - Auto-format code with ruff"
@echo " make test - Run tests with coverage"
@echo " make test - Run Python tests (pytest)"
@echo " make qt - Run quick tests only (exclude slow/benchmark)"
@echo " make cpp-test - Build and run C++ tests"
@echo " make cov - Coverage summary + XML + single-page combined HTML"
Expand Down Expand Up @@ -281,6 +281,7 @@ cov:
--object-directory build/cpp-tests-cov \
--filter 'include/netgraph' --filter 'src' --exclude 'tests' --exclude 'bindings/.*' --exclude '.*pybind11.*' --exclude '_deps/pybind11-src/.*' \
--gcov-ignore-errors=all \
--gcov-ignore-parse-errors=negative_hits.warn_once_per_file \
--xml-pretty -o build/coverage/coverage-cpp.xml
@echo ""
@echo "================ Python + C++ coverage (summary) ================"
Expand All @@ -296,7 +297,8 @@ sanitize-test:
if command -v ninja >/dev/null 2>&1; then GEN_ARGS="-G Ninja"; fi; \
cmake -S . -B "$$BUILD_DIR" -DNETGRAPH_CORE_BUILD_TESTS=ON -DNETGRAPH_CORE_SANITIZE=ON -DCMAKE_BUILD_TYPE=Debug $$GEN_ARGS; \
cmake --build "$$BUILD_DIR" --config Debug -j; \
ASAN_OPTIONS=detect_leaks=1 ctest --test-dir "$$BUILD_DIR" --output-on-failure || echo "⚠️ Some sanitizer tests failed"
if [ "$$(uname -s)" = "Darwin" ]; then ASAN_ENV="ASAN_OPTIONS=detect_leaks=0"; else ASAN_ENV="ASAN_OPTIONS=detect_leaks=1"; fi; \
env $$ASAN_ENV ctest --test-dir "$$BUILD_DIR" --output-on-failure || echo "⚠️ Some sanitizer tests failed"

# Clean + reinstall in dev mode (respects CMAKE_ARGS and MACOSX_DEPLOYMENT_TARGET)
# Uses active PYTHON (venv or PATH) to avoid environment mismatches
Expand Down
Loading
Loading