Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 5 additions & 13 deletions documents/LATENCY_WHITEPAPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,20 +134,12 @@ so stride must be a multiple of 8.

The following behaviors are explicitly covered by unit tests:

- **Null buffers, zero/misaligned strides, and one-node buffers are rejected with exact reasons**
(`SetupLatencyChainRejectsInvalidInputsWithExactReasons`)
- **The minimum two-node chain is accepted at pointer-sized and default strides**
(`SetupLatencyChainAcceptsMinimumTwoNodeChainsAtSupportedStrides`)
- **A constructed chain visits every aligned in-buffer node exactly once and returns to its head**
(`SetupLatencyChainCreatesValidChain`)
- **Pointer count, touched-page count, page size, and stride diagnostics are exact**
(`SetupLatencyChainCollectsDiagnostics`)
- **A locality window that cannot contain two stride-spaced nodes is rejected**
(`SetupLatencyChainWithTooSmallTlbLocalityFails`)
- **A locality-using box mode with a zero locality window is rejected**
(`SetupLatencyChainWithBoxModeAndZeroLocalityFails`)
- **Null buffers, zero/misaligned strides, one-node buffers, unusable locality windows, and box modes without
locality are rejected with exact reasons** (`SetupLatencyChainRejectsInvalidInputsWithExactReasons`).
- **Two-node and four-node chains at pointer-sized and default strides visit every aligned in-buffer node exactly
once and return to the head** (`SetupLatencyChainCreatesValidChain`).
- **Equal explicit seeds reproduce the chain, different seeds change it, and same-random box mode reuses one
permutation across boxes** (`ExplicitSeedsAreReproducibleAndSameRandomModeReusesBoxPermutation`)
permutation across boxes** (`ExplicitSeedsAreReproducibleAndSameRandomModeReusesBoxPermutation`).

These tests validate setup correctness and failure handling before timing is run.

Expand Down
3 changes: 1 addition & 2 deletions documents/LLM_MEMORY_PROFILE_WHITEPAPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -793,8 +793,7 @@ The maintained examples are in [CPU kernel tests](../tests/test_llm_memory_kerne
[Metal helper tests](../tests/test_llm_metal_checksum.mm) for
`SharedAffineLanesExposeBoundedContentCollisions`. The
[Metal backend tests](../tests/test_llm_metal_backend.cpp) contain
`DecodePagedPermutationAndPaddingHooksAreDetectedIntegration` and
`PrefillPagedPermutationAndPaddingHooksAreDetectedIntegration`. Their real table/blit mutations differ from
`PagedPermutationAndPaddingHooksAreDetectedIntegration`, which covers decode and prefill. Its real table/blit mutations differ from
`force_timed_checksum_mismatch` (host readback altered after execution) and `force_kv_write_validation_mismatch`
(forced validation boolean), which test result handling. Pure independent checksum-oracle goldens test arithmetic;
they are not GPU execution evidence. Algorithms and their existing identities remain unchanged by this fault model.
Expand Down
11 changes: 5 additions & 6 deletions documents/PROJECT_STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ Core infrastructure for configuration, memory management, macOS system introspec
| `buffer_manager.h` | Defines the phase-local `BenchmarkBuffers` and command-local `PatternBuffers` RAII owners |
| `buffer_allocator.h` / `.cpp` | Atomically allocates the pattern source/destination pair and performs overflow-safe peak-concurrent memory accounting |
| `buffer_initializer.h` / `.cpp` | Initializes deterministic pattern source data and the zeroed pattern destination before execution |
| `memory_utils.h` / `.cpp` | Cache-line alignment helpers, deterministic/unseeded latency-chain construction and diagnostics, and basic buffer initialization |
| `memory_utils.h` / `.cpp` | Cache-line alignment helpers, deterministic/unseeded latency-chain construction, and basic buffer initialization |

#### src/core/system/

Expand Down Expand Up @@ -375,20 +375,19 @@ installed. All test `.cpp` and `.mm` files are picked up automatically by the Ma
|---|---|---|
| `test_config.cpp` | `ConfigTest` | Strict whole-token CLI/sweep parsing, validation, defaults, and derived buffer/access math |
| `test_signal_handler.cpp` | `BenchmarkSignalMaskGuardTest` | Exact thread-mask restoration, caller-preserved blocked signals, and nested scope ownership |
| `test_messages.cpp` | `Messages*Test` | Console message string functions across all categories |
| `test_messages.cpp` | `Messages*Test` | Exact help and formatted console message contracts |
| `test_memory_utils.cpp` | `MemoryUtilsTest` | Memory helpers: pointer-chase chain construction and verification |
| `test_memory_manager.cpp` | `MemoryManagerTest` | Injected mmap/madvise policy, failures, and exact RAII unmapping |
| `test_numeric_utils.cpp` | `NumericUtilsTest` | Overflow-safe arithmetic, duration calibration, pilot counts, and quantization boundaries |
| `test_llm_memory_contract.cpp` | `LlmMemoryContractTest` | Independent executable specification for all eight decode/prefill backend/layout methodology identities, payloads, paged geometry/lookup goldens, append and full-prompt writes, checksum/event-trace identities, descriptor ABI layouts, and schema acceptance; it does not exercise production LLM code |
| `test_llm_memory_config.cpp` | `LlmMemoryConfigTest` | Config/status defaults, exact standalone decode/prefill parsing including paged prefill, phase/layout rules and stable reasons, strict decimal errors, raw output values, help isolation, worker/seed resolution, geometry/work-limit preflight, and incompatible options |
| `test_llm_memory_config.cpp` | `LlmMemoryConfigTest` | Exact standalone decode/prefill parsing including paged prefill, phase/layout rules and stable reasons, strict decimal errors, raw output values, help isolation, worker/seed resolution, geometry/work-limit preflight, and incompatible options |
| `test_llm_memory_work_plan.cpp` | `LlmMemoryWorkPlanTest` | Production checked decode/prefill geometry and payloads, physical/padding/table/lookup/accounted math including paged-prefill `N+2*M`, deterministic permutation/hash, CPU token/block ownership, Metal cyclic block-owner scheduling and component identities, memory budget, descriptor/range/layout invariants, worker reduction, scenario caps/calibration, and cyclic order |
| `test_llm_memory_executor.cpp` | `LlmMemoryExecutorTest` | CPU phase/layout dispatch, atomic mapping/table-preparation failure seams, contiguous and paged decode/prefill descriptor materialization, independent checksums, padding canaries, synchronized timing, QoS, cancellation, and fake-kernel validation |
| `test_llm_memory_runner.cpp` | `LlmMemoryRunnerTest` | Fake-backend lifecycle and generic decode/prefill task seams, lifecycle unsupported/failure handling, common identity/timing/completion/validation acceptance, exact calibration/single-unit/freeze/frozen-warmup order, cyclic measurements, status/counter/aggregate semantics, interruption, release/checkpoint precedence, auxiliary budgeting, and runner exception boundaries |
| `test_llm_memory_json.cpp` | `LlmMemoryJsonTest` | Schema-2 identities for all eight profiles, canonical plan references, expected/actual checksums, named validation, accepted populations, raw Mach timing, build manifest, nullable geometry/workers, paged and Metal resource/grid evidence, output-peak estimates, acceptance, environment, and bounded checkpoint observations |
| `test_llm_metal_checksum.mm` | `LlmMetalChecksumHelperIntegrationTest` | Real-device checks of shared MSL affine checksum collision boundaries and six-component modulo reduction across partial SIMD/threadgroups |
| `test_llm_memory_output.cpp` | `LlmMemoryOutputTest` | Exact decode/prefill, contiguous/paged, and Metal device/resource/table/task K/V-write/lookup/padding plus cyclic owner/per-threadgroup cost formatting, interpretation text, and deduplicated warnings |
| `test_llm_memory_kernels.cpp` | `LlmMemoryKernelIntegrationTest` | Real contiguous/paged decode/prefill ARM64 descriptor/kernel scenarios, paged tails/lookups, full-prompt writes, exact partial tiled scans, checksum, padding, worker-count, and AAPCS64 coverage |
| `test_llm_metal_backend.cpp` | `LlmMetalBackendTest`, `LlmMetalBackendIntegrationTest`, `LlmMetalBackendFailureInjectionIntegrationTest` | Pure capability, contiguous/paged segmentation, argument-buffer, decode/prefill ABIs, source-hash and prefill loop-order audit, phase-aware checksum/reason mapping, paged-prefill `N+2*M`, cyclic no-duplication ownership and per-threadgroup accounted-byte evidence, plus real-device selected-profile compilation, Tier-2 slots, private table/K/V initialization, exact-tail decode, full-prompt/tiled-prefix prefill, final-ordinal write/padding detection, interruption/failure cleanup, and idempotent release |
| `test_llm_metal_backend.cpp` | `LlmMetalBackendTest`, `LlmMetalBackendIntegrationTest`, `LlmMetalBackendFailureInjectionIntegrationTest` | Pure capability, contiguous/paged segmentation, argument-buffer, decode/prefill ABIs, source provenance and prefill loop-order audit, phase-aware checksum/reason mapping, paged-prefill `N+2*M`, cyclic no-duplication ownership and per-threadgroup accounted-byte evidence, plus real-device selected-profile compilation, Tier-2 slots, private table/K/V initialization, exact-tail decode, full-prompt/tiled-prefix prefill, final-ordinal write/padding detection, interruption/failure cleanup, and idempotent release |
| `test_buffer_manager.cpp` | `BufferManagerTest` | Pattern mapping policy, atomic allocation cleanup, initialized content, validation, and peak accounting |
| `test_benchmark_executor.cpp` | `BenchmarkExecutorTest` | Injected phase/chain failures, continuous latency sampling, and hardware executor contracts |
| `test_benchmark_runner.cpp` | `BenchmarkStatisticsCollectorTest`, `BenchmarkRunnerTest` | Status-bearing aggregation, schema-3 retained snapshots, checkpointing, interruption, and runner exception/failure seams |
Expand All @@ -403,7 +402,7 @@ installed. All test `.cpp` and `.mm` files are picked up automatically by the Ma
| `test_llm_result_verifier.py` | Independent LLM verifier | Eight current profiles, arithmetic goldens, status/population and semantic mutations, bounded input and build provenance |
| `test_script_examples.py` | Python script-example entry test | Compatible standard schema-3/methodology inputs, provenance-version handling, completion and field-shape checks, and metric extraction for the bundled standard-memory examples, including the optional jq-backed shell path when jq is installed |
| `test_json_output_session.cpp` | `JsonOutputTargetTest`, `JsonOutputSessionTest` | Exact sentinel/path classification, lazy checkpoint dispatch, atomic-file parity, stdout routing/restoration, and stream failure containment |
| `test_json_utils.cpp` | `JsonUtilsTest`, `JsonFileWriterTest` | JSON parse/statistics and atomic writer success/failure contracts |
| `test_json_utils.cpp` | `JsonUtilsTest`, `JsonFileWriterTest` | UTC timestamp/statistics and atomic writer success/failure contracts |
| `test_output_printer.cpp` | `OutputPrinterTest`, `OutputPrinterCustomCacheUnitsTest` | Status-aware partial output, mode/cache composition, and custom-cache size-unit boundaries |
| `test_sweep_runner.cpp` | `SweepRunnerTest` | Current-only nested standard schema-3 completion classification, complete/partial/interrupted/failed attempt accounting, schema-1 envelopes, and file/lazy-stdout checkpoint behavior |
| `test_sweep_utils.cpp` | `SweepUtilsTest` | Shared sweep parsing, empty-dimension behavior, and overflow-safe Cartesian counts |
Expand Down
2 changes: 1 addition & 1 deletion documents/TECHNICAL_SPECIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1482,7 +1482,7 @@ Recommended validation commands:
- GPU help smoke check: `./memory_benchmark --gpu-bandwidth --help`
- LLM help smoke check: `./memory_benchmark --llm-memory --help`
- Deterministic LLM tests:
`./test_runner '--gtest_filter=ModeSelectorTest.*:LlmMemoryContractTest.*:LlmMemoryConfigTest.*:LlmMemoryWorkPlanTest.*:LlmMemoryExecutorTest.*:LlmMemoryRunnerTest.*:LlmMemoryJsonTest.*:LlmMemoryOutputTest.*:MessagesTest.*'`
`./test_runner '--gtest_filter=ModeSelectorTest.*:LlmMemoryConfigTest.*:LlmMemoryWorkPlanTest.*:LlmMemoryExecutorTest.*:LlmMemoryRunnerTest.*:LlmMemoryJsonTest.*:LlmMemoryOutputTest.*:Messages*Test.*'`
- Real LLM ARM64 and bounded executable transport contracts:
`./test_runner '--gtest_filter=*LlmMemory*Integration*:*ExecutableCliIntegration*'`
- LLM Metal decode and prefill for contiguous and paged KV:
Expand Down
15 changes: 1 addition & 14 deletions src/benchmark/benchmark_executor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,6 @@ int prepare_cache_latency_buffers(BenchmarkConfig& config, BenchmarkBuffers& buf
config.custom_buffer_size,
config.latency_stride_bytes,
config.latency_tlb_locality_bytes,
nullptr,
config.latency_chain_mode,
derive_benchmark_seed(config.benchmark_seed,
kSeedDomainCustomLatency));
Expand All @@ -303,7 +302,6 @@ int prepare_cache_latency_buffers(BenchmarkConfig& config, BenchmarkBuffers& buf
config.l1_buffer_size,
config.latency_stride_bytes,
config.latency_tlb_locality_bytes,
nullptr,
config.latency_chain_mode,
derive_benchmark_seed(config.benchmark_seed,
kSeedDomainL1Latency)) != EXIT_SUCCESS) {
Expand All @@ -321,7 +319,6 @@ int prepare_cache_latency_buffers(BenchmarkConfig& config, BenchmarkBuffers& buf
config.l2_buffer_size,
config.latency_stride_bytes,
config.latency_tlb_locality_bytes,
nullptr,
config.latency_chain_mode,
derive_benchmark_seed(config.benchmark_seed,
kSeedDomainL2Latency)) != EXIT_SUCCESS) {
Expand Down Expand Up @@ -352,7 +349,6 @@ int prepare_main_memory_latency_buffer(BenchmarkConfig& config, BenchmarkBuffers
config.buffer_size,
config.latency_stride_bytes,
config.latency_tlb_locality_bytes,
nullptr,
config.latency_chain_mode,
derive_benchmark_seed(config.benchmark_seed,
kSeedDomainMainLatency));
Expand Down Expand Up @@ -443,8 +439,7 @@ void run_paired_locality_comparison(void* buffer,
const LatencyChainMode mode = use_locality
? LatencyChainMode::RandomInBoxRandomBox
: LatencyChainMode::GlobalRandom;
if (setup_latency_chain(buffer, buffer_size, stride_bytes, locality_bytes,
nullptr, mode, seed) != EXIT_SUCCESS) {
if (setup_latency_chain(buffer, buffer_size, stride_bytes, locality_bytes, mode, seed) != EXIT_SUCCESS) {
return false;
}
show_progress();
Expand Down Expand Up @@ -1047,14 +1042,6 @@ BenchmarkResults run_single_benchmark_loop(BenchmarkConfig& config,
throw std::runtime_error(
Messages::benchmark_reason_prepare_failed(enabled_phase.name));
}
if (test_hooks != nullptr && test_hooks->fail_latency_chain_setup &&
(enabled_phase.phase == Phase::CacheLatency ||
enabled_phase.phase == Phase::MainLatency) &&
test_hooks->fail_latency_chain_setup(enabled_phase.name)) {
throw std::runtime_error(
Messages::benchmark_reason_latency_chain_setup_failed(
enabled_phase.name));
}
switch (enabled_phase.phase) {
case Phase::MainBandwidth: {
BenchmarkBuffers phase_buffers;
Expand Down
1 change: 0 additions & 1 deletion src/benchmark/benchmark_executor.h
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ struct BenchmarkExecutionState;
/** @brief Optional kernel-free fault seams for phase preparation tests. */
struct BenchmarkExecutorTestHooks {
std::function<bool(const std::string&)> fail_phase_preparation;
std::function<bool(const std::string&)> fail_latency_chain_setup;
};

/**
Expand Down
2 changes: 0 additions & 2 deletions src/core/config/constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,6 @@ namespace Constants {
"llm-kv-layout-geometry-v1";
constexpr const char* LLM_KV_LAYOUT_PLAN_IDENTITY_VERSION =
"llm-kv-layout-plan-v1";
constexpr const char* LLM_PAGED_DECODE_WORKLOAD_IDENTITY_VERSION =
"llm-paged-decode-workload-v1";
constexpr const char* LLM_PAGED_CPU_EXECUTION_IDENTITY_VERSION =
"llm-paged-cpu-execution-v1";
constexpr const char* LLM_PAGED_METAL_EXECUTION_IDENTITY_VERSION =
Expand Down
Loading