From 263673c97b0921a2912f83b56483422a1d192a27 Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 07:12:39 -0400 Subject: [PATCH 1/9] Fix PHP 8.6 sapi_globals offset and PHP 7.0 frame-slot bug; add arm struct probe structs/x86_64/php_structs_86.h: sapi_globals_struct_86.global_request_time was declared at offset 440; real PHP 8.6.0beta2 has it at 432 (verified via gdb/offsetof). Before: `-r` reported `# ts = 0.000000` on 8.6. After: a correct wall-clock timestamp. phpspy_trace.c/phpspy_trace_tpl.c/structs/structs.h: the CV frame-slot constant used by --peek-var was hardcoded as the literal 5. The real value is ZEND_CALL_FRAME_SLOT = ceil(sizeof(zend_execute_data)/sizeof(zval)), which is 6 on PHP 7.0 (it still carries execute_data.called_scope, removed in 7.1) and 5 on every later version -- verified against all 12 supported versions' real headers. Before: --peek-var silently found nothing on 7.0. After: tests/test_varpeek.sh passes on 7.0; every other version unaffected. struct_dump.sh/struct_dump.gdb: bring the dump tooling current -- add the missing php-8.4.25/php-8.5.10 tags (it stopped at 8.3/master), add an optional single-version argument so a CI matrix cell can dump just one version, and add sizeof(zend_execute_data)/sizeof(zval) cross-check lines for the frame-slot fix above. .github/workflows/dump_structs.yml: new, workflow_dispatch-only job that builds php-src from source for every supported version on both x86_64 and aarch64 and uploads the raw struct_dump.gdb output as artifacts -- this is how real (not placeholder/guessed) aarch64 offsets get produced; see CLAUDE.md and the phpspy CLAUDE.md for the struct-mirror background. Verified: full test suite passes unchanged across all 13 locally installed PHP versions (70-86, excluding asan) before and after this change; the only failures present (pdo_args_packed_array on 72/73/74/80/81/82) are a pre-existing, unrelated HASH_FLAG_PACKED bug, confirmed identical via git stash before committing. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- .github/workflows/dump_structs.yml | 63 ++++++++++++++++++++++++++++++ phpspy.h | 2 +- phpspy_trace.c | 7 +++- phpspy_trace_tpl.c | 12 ++++++ struct_dump.gdb | 7 ++++ struct_dump.sh | 38 ++++++++++++------ structs/structs.h | 3 ++ structs/x86_64/php_structs_86.h | 4 +- 8 files changed, 120 insertions(+), 16 deletions(-) create mode 100644 .github/workflows/dump_structs.yml diff --git a/.github/workflows/dump_structs.yml b/.github/workflows/dump_structs.yml new file mode 100644 index 0000000..d91ab78 --- /dev/null +++ b/.github/workflows/dump_structs.yml @@ -0,0 +1,63 @@ +name: dump_structs + +# One-off, manually-triggered probe: builds php-src from source for every PHP +# version phpspy ships struct mirrors for, on both x86_64 and aarch64, and +# uploads the raw struct_dump.gdb output as build artifacts. This is how real +# (not guessed) aarch64 offsets get produced -- see struct_dump.sh/.gdb and +# CLAUDE.md. Not run on push/pull_request: 12 versions x 2 arches, each a full +# ./buildconf && ./configure && make, is far too slow for routine CI. + +on: + workflow_dispatch: + +jobs: + dump: + strategy: + fail-fast: false + matrix: + runs_on: [ubuntu-24.04, ubuntu-24.04-arm] + phpv: + - php-7.0.33 + - php-7.1.33 + - php-7.2.34 + - php-7.3.33 + - php-7.4.33 + - php-8.0.30 + - php-8.1.28 + - php-8.2.18 + - php-8.3.6 + - php-8.4.25 + - php-8.5.10 + - master + + runs-on: ${{ matrix.runs_on }} + + steps: + - name: checkout phpspy + uses: actions/checkout@v4 + with: + path: phpspy + + - name: checkout php-src + uses: actions/checkout@v4 + with: + repository: php/php-src + path: php-src + fetch-depth: 0 + + - name: deps + run: | + sudo apt-get update + sudo apt-get install -y \ + autoconf bison re2c pkg-config build-essential gdb \ + libxml2-dev libsqlite3-dev libssl-dev + + - name: dump + run: ./phpspy/struct_dump.sh "$GITHUB_WORKSPACE/php-src" "${{ matrix.phpv }}" + + - name: upload + uses: actions/upload-artifact@v4 + with: + name: struct_dump-${{ matrix.runs_on }}-${{ matrix.phpv }} + path: phpspy/struct_dump.${{ matrix.phpv }}.out + if-no-files-found: warn diff --git a/phpspy.h b/phpspy.h index 16fc0f0..dcc66cb 100644 --- a/phpspy.h +++ b/phpspy.h @@ -44,7 +44,7 @@ #define STR1(s) #s #define STR2(s) STR1(s) -#define PHPSPY_VERSION "0.8.0" +#define PHPSPY_VERSION "0.8.1" #define PHPSPY_MIN(a, b) ((a) < (b) ? (a) : (b)) #define PHPSPY_MAX(a, b) ((a) > (b) ? (a) : (b)) #ifndef PHPSPY_STR_SIZE diff --git a/phpspy_trace.c b/phpspy_trace.c index 2711d22..40285ba 100644 --- a/phpspy_trace.c +++ b/phpspy_trace.c @@ -340,8 +340,11 @@ static int trace_locals(trace_context *context, zend_op *zop, zend_execute_data HASH_FIND(hh, entry->varmap, tmp, tmp_len, var); if (!var) continue; num_vars_found += 1; - /* See ZEND_CALL_VAR_NUM macro in php-src */ - try_copy_proc_mem("zval", ((zval*)(remote_execute_data)) + ((int)(5 + i)), &zv, sizeof(zv)); + /* See ZEND_CALL_VAR_NUM macro in php-src. The frame-slot count is + version-dependent (PHP 7.0 differs from every later version); see + phpspy_frame_slot, defined per-phpv in phpspy_trace_tpl.c and for + USE_ZEND in structs/structs.h. */ + try_copy_proc_mem("zval", ((zval*)(remote_execute_data)) + ((int)(phpspy_frame_slot + i)), &zv, sizeof(zv)); try(rv, sprint_zval(context, &zv, tmp, sizeof(tmp), &tmp_len)); context->event.varpeek.entry = entry; context->event.varpeek.var = var; diff --git a/phpspy_trace_tpl.c b/phpspy_trace_tpl.c index f1ff746..e6fd374 100644 --- a/phpspy_trace_tpl.c +++ b/phpspy_trace_tpl.c @@ -39,6 +39,17 @@ #define sprint_pdo_binds concat2(sprint_pdo_binds_, phpv) #define sprint_pdo_bind concat2(sprint_pdo_bind_, phpv) +/* ZEND_CALL_FRAME_SLOT = ceil(sizeof(zend_execute_data) / sizeof(zval)). Measured + directly against every supported PHP version's real headers: 6 on 7.0 (which + still carries execute_data.called_scope, removed in 7.1), 5 on every later + version (sizeof(zend_execute_data) is 72 or 80 there, both of which round up + to 5 slots of 16 bytes). */ +#if phpv == 70 +#define phpspy_frame_slot 6 +#else +#define phpspy_frame_slot 5 +#endif + #include "phpspy_trace.c" #undef concat1 @@ -73,6 +84,7 @@ #undef trace_pdo #undef sprint_pdo_binds #undef sprint_pdo_bind +#undef phpspy_frame_slot #undef copy_executor_globals #undef copy_zarray_bucket #undef sprint_zstring diff --git a/struct_dump.gdb b/struct_dump.gdb index 10072fc..0ec8999 100644 --- a/struct_dump.gdb +++ b/struct_dump.gdb @@ -95,6 +95,13 @@ whatis zval fieldof zval u2.next printf "\n" +# Cross-check for phpspy_frame_slot (phpspy_trace_tpl.c / structs/structs.h): +# ZEND_CALL_FRAME_SLOT = ceil(sizeof(zend_execute_data) / sizeof(zval)). +printf "frame_slot\n" +printf " sizeof(zend_execute_data) %lu\n", sizeof(zend_execute_data) +printf " sizeof(zval) %lu\n", sizeof(zval) +printf "\n" + printf "Bucket\n" whatis Bucket fieldof Bucket val diff --git a/struct_dump.sh b/struct_dump.sh index 5ce2b61..a66afc8 100755 --- a/struct_dump.sh +++ b/struct_dump.sh @@ -1,28 +1,44 @@ #!/bin/bash this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) phpsrc_dir=$1 +only_phpv=$2 if [ -z "$phpsrc_dir" ]; then echo "Required: php-src directory" exit 1 fi +# All PHP versions phpspy ships struct mirrors for. 8.6 has no release tag yet +# (still dev), so it stays pinned to `master`, same as when the 8.6 structs +# were first cut. +all_phpvs=( + php-7.0.33 + php-7.1.33 + php-7.2.34 + php-7.3.33 + php-7.4.33 + php-8.0.30 + php-8.1.28 + php-8.2.18 + php-8.3.6 + php-8.4.25 + php-8.5.10 + master +) + +if [ -n "$only_phpv" ]; then + phpvs=("$only_phpv") +else + phpvs=("${all_phpvs[@]}") +fi + pushd "$phpsrc_dir" || exit 1 git fetch --tags -for phpv in php-7.0.33 \ - php-7.1.33 \ - php-7.2.34 \ - php-7.3.33 \ - php-7.4.33 \ - php-8.0.30 \ - php-8.1.28 \ - php-8.2.18 \ - php-8.3.6 \ - master +for phpv in "${phpvs[@]}" do git reset --hard HEAD \ && git clean -fdx \ - && git checkout $phpv \ + && git checkout "$phpv" \ && git clean -fdx \ && ./buildconf --force \ && ./configure \ diff --git a/structs/structs.h b/structs/structs.h index b6daf99..3dba6f3 100644 --- a/structs/structs.h +++ b/structs/structs.h @@ -6,6 +6,9 @@ # undef snprintf # undef vsnprintf # undef HASH_ADD + /* ZEND_CALL_FRAME_SLOT (Zend/zend_compile.h) is already visible here via + main/SAPI.h's own includes; no extra #include is needed. */ +# define phpspy_frame_slot ZEND_CALL_FRAME_SLOT #else # if defined(__x86_64__) # include diff --git a/structs/x86_64/php_structs_86.h b/structs/x86_64/php_structs_86.h index bf1cf7f..723fc94 100644 --- a/structs/x86_64/php_structs_86.h +++ b/structs/x86_64/php_structs_86.h @@ -119,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_86 { struct __attribute__((__packed__)) _sapi_globals_struct_86 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_86 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[376]; /* 56 +376 */ + double global_request_time; /* 432 +8 */ }; struct __attribute__((__packed__)) _Bucket_86 { From 33869f28dad0ddbade900f702f91ba80215232a1 Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 07:35:28 -0400 Subject: [PATCH 2/9] Don't bump PHPSPY_VERSION for this fix Leave version bumps to the maintainer's judgment on when a new version is warranted. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- phpspy.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/phpspy.h b/phpspy.h index dcc66cb..16fc0f0 100644 --- a/phpspy.h +++ b/phpspy.h @@ -44,7 +44,7 @@ #define STR1(s) #s #define STR2(s) STR1(s) -#define PHPSPY_VERSION "0.8.1" +#define PHPSPY_VERSION "0.8.0" #define PHPSPY_MIN(a, b) ((a) < (b) ? (a) : (b)) #define PHPSPY_MAX(a, b) ((a) > (b) ? (a) : (b)) #ifndef PHPSPY_STR_SIZE From 135e4732a3bc0b52ee53a48c16dde60937b34fe6 Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 07:35:45 -0400 Subject: [PATCH 3/9] Replace guessed/placeholder aarch64 structs with real probed data structs/aarch64/php_structs_{70,71,72,73,74,80,81,82}.h were byte-identical copies of their x86_64 counterparts with a "these structs are wrong for aarch64" comment -- never actually measured on arm. 83/84/85 carried one real but unverified guess (a 16-byte shrink in sapi_globals_struct, since zend_stat_t/struct stat is 128 bytes on aarch64/glibc vs 144 on x86_64/glibc); 86 carried a guess that was wrong given the x86_64 fix in the prior commit. Derived via the new dump_structs.yml workflow (workflow_dispatch): it builds php-src from source for every supported version on both ubuntu-24.04 and ubuntu-24.04-arm and runs the (now current) struct_dump.gdb against each resulting binary, uploading the raw offsets as build artifacts. Run at https://github.com/rlerdorf/phpspy/actions/runs/34219729971 (all 24 cells succeeded). Before touching any header, the x86_64 leg of that same run was diffed field-for-field against this session's own independently-gathered x86_64 ground truth (gdb/offsetof against the locally installed PHP 7.0-8.6) -- byte-identical across all 12 versions, confirming the build+dump pipeline itself is trustworthy. The aarch64 leg then showed a single, consistent difference from x86_64 for every version: sapi_globals_struct's global_request_time shifts by exactly -16 bytes, matching the zend_stat_t size delta above, and nothing else in the struct surface phpspy reads differs by architecture. That means: - 83 and 85's existing guessed delta was correct -- confirmed against real hardware, no change needed. - 70-74/80-82 needed the same delta, applied for the first time. - 86 needed the new delta following the corrected x86_64 offset from the prior commit (432 - 16 = 416, not the previously-guessed 424). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- structs/aarch64/php_structs_70.h | 7 ++----- structs/aarch64/php_structs_71.h | 7 ++----- structs/aarch64/php_structs_72.h | 7 ++----- structs/aarch64/php_structs_73.h | 7 ++----- structs/aarch64/php_structs_74.h | 7 ++----- structs/aarch64/php_structs_80.h | 7 ++----- structs/aarch64/php_structs_81.h | 7 ++----- structs/aarch64/php_structs_82.h | 7 ++----- structs/aarch64/php_structs_86.h | 4 ++-- 9 files changed, 18 insertions(+), 42 deletions(-) diff --git a/structs/aarch64/php_structs_70.h b/structs/aarch64/php_structs_70.h index dafdeee..11fb6b3 100644 --- a/structs/aarch64/php_structs_70.h +++ b/structs/aarch64/php_structs_70.h @@ -1,8 +1,6 @@ #ifndef __php_structs_70_h #define __php_structs_70_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_70 zend_executor_globals_70; @@ -122,8 +120,8 @@ struct __attribute__((__packed__)) _sapi_request_info_70 { struct __attribute__((__packed__)) _sapi_globals_struct_70 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_70 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_70 { @@ -142,7 +140,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_70 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_70 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_70 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_71.h b/structs/aarch64/php_structs_71.h index f7b8b28..6bbe8f1 100644 --- a/structs/aarch64/php_structs_71.h +++ b/structs/aarch64/php_structs_71.h @@ -1,8 +1,6 @@ #ifndef __php_structs_71_h #define __php_structs_71_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_71 zend_executor_globals_71; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_71 { struct __attribute__((__packed__)) _sapi_globals_struct_71 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_71 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_71 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_71 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_71 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_71 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_72.h b/structs/aarch64/php_structs_72.h index 3e7729b..0670045 100644 --- a/structs/aarch64/php_structs_72.h +++ b/structs/aarch64/php_structs_72.h @@ -1,8 +1,6 @@ #ifndef __php_structs_72_h #define __php_structs_72_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_72 zend_executor_globals_72; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_72 { struct __attribute__((__packed__)) _sapi_globals_struct_72 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_72 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_72 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_72 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_72 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_72 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_73.h b/structs/aarch64/php_structs_73.h index 2a02a7c..3cdde01 100644 --- a/structs/aarch64/php_structs_73.h +++ b/structs/aarch64/php_structs_73.h @@ -1,8 +1,6 @@ #ifndef __php_structs_73_h #define __php_structs_73_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_73 zend_executor_globals_73; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_73 { struct __attribute__((__packed__)) _sapi_globals_struct_73 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_73 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_73 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_73 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_73 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_73 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_74.h b/structs/aarch64/php_structs_74.h index c448737..7e526c6 100644 --- a/structs/aarch64/php_structs_74.h +++ b/structs/aarch64/php_structs_74.h @@ -1,8 +1,6 @@ #ifndef __php_structs_74_h #define __php_structs_74_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_74 zend_executor_globals_74; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_74 { struct __attribute__((__packed__)) _sapi_globals_struct_74 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_74 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_74 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_74 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_74 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_74 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_80.h b/structs/aarch64/php_structs_80.h index 20d19da..ddb40b1 100644 --- a/structs/aarch64/php_structs_80.h +++ b/structs/aarch64/php_structs_80.h @@ -1,8 +1,6 @@ #ifndef __php_structs_80_h #define __php_structs_80_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_80 zend_executor_globals_80; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_80 { struct __attribute__((__packed__)) _sapi_globals_struct_80 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_80 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_80 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_80 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_80 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_80 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_81.h b/structs/aarch64/php_structs_81.h index 6953dc4..484e81d 100644 --- a/structs/aarch64/php_structs_81.h +++ b/structs/aarch64/php_structs_81.h @@ -1,8 +1,6 @@ #ifndef __php_structs_81_h #define __php_structs_81_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_81 zend_executor_globals_81; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_81 { struct __attribute__((__packed__)) _sapi_globals_struct_81 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_81 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_81 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_81 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_81 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_81 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_82.h b/structs/aarch64/php_structs_82.h index dd7f0c5..1db3e48 100644 --- a/structs/aarch64/php_structs_82.h +++ b/structs/aarch64/php_structs_82.h @@ -1,8 +1,6 @@ #ifndef __php_structs_82_h #define __php_structs_82_h -/* TODO: These structs are wrong for aarch64 */ - #include typedef struct _zend_executor_globals_82 zend_executor_globals_82; @@ -121,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_82 { struct __attribute__((__packed__)) _sapi_globals_struct_82 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_82 request_info; /* 8 +48 */ - uint8_t pad1[384]; /* 56 +384 */ - double global_request_time; /* 440 +8 */ + uint8_t pad1[368]; /* 56 +368 */ + double global_request_time; /* 424 +8 */ }; struct __attribute__((__packed__)) _Bucket_82 { @@ -141,7 +139,6 @@ struct __attribute__((__packed__)) _zend_mm_heap_82 { size_t peak; /* 24 +8 */ }; - struct __attribute__((__packed__)) _zend_object_82 { uint8_t pad0[16]; /* 0 +16 */ zend_class_entry_82 *ce; /* 16 +8 */ diff --git a/structs/aarch64/php_structs_86.h b/structs/aarch64/php_structs_86.h index 3db5b59..2961e9e 100644 --- a/structs/aarch64/php_structs_86.h +++ b/structs/aarch64/php_structs_86.h @@ -119,8 +119,8 @@ struct __attribute__((__packed__)) _sapi_request_info_86 { struct __attribute__((__packed__)) _sapi_globals_struct_86 { uint8_t pad0[8]; /* 0 +8 */ sapi_request_info_86 request_info; /* 8 +48 */ - uint8_t pad1[368]; /* 56 +368 */ - double global_request_time; /* 424 +8 */ + uint8_t pad1[360]; /* 56 +360 */ + double global_request_time; /* 416 +8 */ }; struct __attribute__((__packed__)) _Bucket_86 { From 4960663dd7c5126f247b69663dd82a9a3e4ba07c Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 08:32:54 -0400 Subject: [PATCH 4/9] Automate struct offset maintenance without a manifest/generator system Adds two scheduled workflows that build on the dump_structs.yml approach already proven this session (real from-source builds on both x86_64 and aarch64, rather than a compile-time manifest/offset-table system): - check_struct_versions.yml (weekly): tools/find_php_updates.sh does a cheap `git ls-remote` check (no PHP build, ~0.5s) for newer tags on any PHP series phpspy already tracks, or for PHP 8.6 finally getting a real release tag instead of `master`. Only when it finds something does it dump the single changed version on both arches, patch the existing mirror headers via tools/render_struct_update.sh, and open a PR -- never auto-merges. A genuinely new PHP series beyond what phpspy supports is deliberately left alone (needs code changes beyond struct data) and only noted on stderr. - struct_drift_check.yml (monthly): reuses dump_structs.yml's full 24-cell matrix (now callable via workflow_call) to re-verify every already- pinned version still matches a fresh build, and opens an issue (not a PR) if anything unexpectedly drifted. tools/patch_struct_header.awk is the core piece: it adjusts only a field whose offset moved plus its immediately preceding pad, using each field's already-absolute offset comment rather than tracking a cursor -- see its header comment for why that's sufficient and what it deliberately refuses to touch (union blocks, fields with no adjustable preceding pad, or a negative resulting pad size all get flagged for a human instead). Verified before committing: - patch_struct_header.awk reproduces both real fixes from the prior commits (x86_64/86, aarch64/86) and all 8 aarch64 placeholder replacements byte-for-byte against the actual committed headers. - Idempotent across all 24 current header/arch combinations (already- correct data produces zero diff). - All four refusal paths (no preceding pad, union field, negative pad, exact match) verified against synthetic fixtures. - find_php_updates.sh run against the real php-src remote. - The full monthly drift-check loop simulated locally against all 24 real CI dump artifacts from this session's earlier dump_structs.yml run: zero drift on current data, and correctly detects+reports drift when fed the known-broken pre-fix 8.6 header. - dump_structs.yml's matrix now reads tools/list_pinned_tags.sh instead of a second hardcoded copy of the version list, so the two can no longer drift apart from each other. Not yet verified: the actual GitHub Actions machinery itself (workflow_call chaining, matrix-from-dynamic-JSON, gh pr/issue create) -- only simulated locally so far. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- .github/workflows/check_struct_versions.yml | 159 ++++++++++++++++++++ .github/workflows/dump_structs.yml | 28 ++-- .github/workflows/struct_drift_check.yml | 82 ++++++++++ struct_dump.sh | 6 + tools/find_php_updates.sh | 51 +++++++ tools/list_pinned_tags.sh | 18 +++ tools/parse_struct_dump.awk | 25 +++ tools/patch_struct_header.awk | 151 +++++++++++++++++++ tools/render_struct_update.sh | 43 ++++++ tools/update_pinned_tag.sh | 30 ++++ 10 files changed, 580 insertions(+), 13 deletions(-) create mode 100644 .github/workflows/check_struct_versions.yml create mode 100644 .github/workflows/struct_drift_check.yml create mode 100755 tools/find_php_updates.sh create mode 100755 tools/list_pinned_tags.sh create mode 100644 tools/parse_struct_dump.awk create mode 100644 tools/patch_struct_header.awk create mode 100755 tools/render_struct_update.sh create mode 100755 tools/update_pinned_tag.sh diff --git a/.github/workflows/check_struct_versions.yml b/.github/workflows/check_struct_versions.yml new file mode 100644 index 0000000..8572ec1 --- /dev/null +++ b/.github/workflows/check_struct_versions.yml @@ -0,0 +1,159 @@ +name: check_struct_versions + +# Weekly, cheap (git ls-remote only, no PHP build): checks whether any PHP +# series phpspy already tracks has a newer release tag than what's pinned in +# struct_dump.sh, or whether PHP 8.6 (currently pinned to `master`, no +# release tag yet) has finally gotten one. If so, dumps just that version on +# both architectures (tools/find_php_updates.sh already scoped this to real +# changes, so this is normally 0 cells, occasionally 2), patches the +# existing mirror headers with tools/render_struct_update.sh, and opens a PR +# for a human to review -- never auto-merges. +# +# A genuinely new PHP series beyond what phpspy supports (e.g. a future 8.7) +# is deliberately NOT handled here -- see find_php_updates.sh's stderr note. +# That needs code changes beyond struct data (phpspy.c dispatch, -V help +# text, README), which is a job for a human, not this workflow. + +on: + schedule: + - cron: '0 6 * * 1' # every Monday, 06:00 UTC + workflow_dispatch: + +jobs: + detect: + runs-on: ubuntu-24.04 + outputs: + changes: ${{ steps.detect.outputs.changes }} + steps: + - uses: actions/checkout@v4 + - id: detect + run: | + set -euo pipefail + lines=$(./tools/find_php_updates.sh) + echo "$lines" + json=$(printf '%s\n' "$lines" | awk 'NF{printf "{\"nn\":\"%s\",\"tag\":\"%s\"}\n", $1, $2}' | jq -s -c .) + echo "changes=$json" >> "$GITHUB_OUTPUT" + + dump: + needs: detect + if: needs.detect.outputs.changes != '[]' + strategy: + fail-fast: false + matrix: + runs_on: [ubuntu-24.04, ubuntu-24.04-arm] + change: ${{ fromJson(needs.detect.outputs.changes) }} + runs-on: ${{ matrix.runs_on }} + steps: + - name: checkout phpspy + uses: actions/checkout@v4 + with: + path: phpspy + - name: checkout php-src + uses: actions/checkout@v4 + with: + repository: php/php-src + path: php-src + fetch-depth: 0 + - name: deps + run: | + sudo apt-get update + sudo apt-get install -y \ + autoconf bison re2c pkg-config build-essential gdb \ + libxml2-dev libsqlite3-dev libssl-dev + - name: dump + env: + TAG: ${{ matrix.change.tag }} + run: ./phpspy/struct_dump.sh "$GITHUB_WORKSPACE/php-src" "$TAG" + - name: upload + env: + TAG: ${{ matrix.change.tag }} + uses: actions/upload-artifact@v4 + with: + name: dump-${{ matrix.runs_on }}-${{ matrix.change.nn }} + path: phpspy/struct_dump.${{ matrix.change.tag }}.out + if-no-files-found: error + + render: + needs: [detect, dump] + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + - name: install gawk + run: sudo apt-get update && sudo apt-get install -y gawk + - uses: actions/download-artifact@v4 + with: + path: dumps + - name: render + env: + CHANGES: ${{ needs.detect.outputs.changes }} + run: | + set -euo pipefail + : > render_status.env + echo "$CHANGES" | jq -c '.[]' | while read -r change; do + nn=$(echo "$change" | jq -r .nn) + tag=$(echo "$change" | jq -r .tag) + tools/update_pinned_tag.sh "$tag" || echo "STATUS_${nn}_pin=failed" >> render_status.env + for pair in "ubuntu-24.04:x86_64" "ubuntu-24.04-arm:aarch64"; do + runs_on=${pair%%:*} + arch=${pair##*:} + raw="dumps/dump-${runs_on}-${nn}/struct_dump.${tag}.out" + if [ ! -f "$raw" ]; then + echo "render: missing dump artifact $raw" >&2 + echo "STATUS_${nn}_${arch}=missing" >> render_status.env + continue + fi + if tools/render_struct_update.sh "$nn" "$arch" "$raw" .; then + echo "STATUS_${nn}_${arch}=ok" >> render_status.env + else + echo "STATUS_${nn}_${arch}=needs_review" >> render_status.env + fi + done + done + cat render_status.env + - name: summarize + id: summary + env: + CHANGES: ${{ needs.detect.outputs.changes }} + run: | + { + echo "body< \(.tag)"' + echo + if grep -q needs_review render_status.env 2>/dev/null; then + echo "**One or more architectures needed manual review and were left" + echo "unchanged -- see the \`render\` job log for exactly which field" + echo "and why.** Do not merge until those are resolved by hand." + fi + echo "STRUCT_PR_BODY_EOF" + } >> "$GITHUB_OUTPUT" + - name: open PR + env: + GH_TOKEN: ${{ github.token }} + PR_BODY: ${{ steps.summary.outputs.body }} + run: | + set -euo pipefail + if git diff --quiet; then + echo "No struct changes to open a PR for." + exit 0 + fi + branch="bot/struct-update-$(date +%Y%m%d)" + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git checkout -b "$branch" + git add structs/ struct_dump.sh + git commit -m "Update struct offsets for newer PHP release(s)" + git push origin "$branch" + title="Update struct offsets for newer PHP release(s)" + if grep -q needs_review render_status.env 2>/dev/null; then + title="[NEEDS REVIEW] $title" + fi + gh pr create --title "$title" --body "$PR_BODY" --head "$branch" diff --git a/.github/workflows/dump_structs.yml b/.github/workflows/dump_structs.yml index d91ab78..6696112 100644 --- a/.github/workflows/dump_structs.yml +++ b/.github/workflows/dump_structs.yml @@ -9,26 +9,28 @@ name: dump_structs on: workflow_dispatch: + workflow_call: jobs: + list: + runs-on: ubuntu-24.04 + outputs: + phpvs: ${{ steps.list.outputs.phpvs }} + steps: + - uses: actions/checkout@v4 + - id: list + run: | + set -euo pipefail + json=$(tools/list_pinned_tags.sh | jq -R -s -c 'split("\n") | map(select(length > 0))') + echo "phpvs=$json" >> "$GITHUB_OUTPUT" + dump: + needs: list strategy: fail-fast: false matrix: runs_on: [ubuntu-24.04, ubuntu-24.04-arm] - phpv: - - php-7.0.33 - - php-7.1.33 - - php-7.2.34 - - php-7.3.33 - - php-7.4.33 - - php-8.0.30 - - php-8.1.28 - - php-8.2.18 - - php-8.3.6 - - php-8.4.25 - - php-8.5.10 - - master + phpv: ${{ fromJson(needs.list.outputs.phpvs) }} runs-on: ${{ matrix.runs_on }} diff --git a/.github/workflows/struct_drift_check.yml b/.github/workflows/struct_drift_check.yml new file mode 100644 index 0000000..208727b --- /dev/null +++ b/.github/workflows/struct_drift_check.yml @@ -0,0 +1,82 @@ +name: struct_drift_check + +# Monthly regression test: re-verifies that the checked-in struct mirrors +# for every already-supported PHP version still match a fresh from-source +# build, on both architectures. This re-dumps the exact tags already pinned +# in struct_dump.sh, so it's not about newer releases -- see +# check_struct_versions.yml for that. It exists to catch the rarer case: +# an upstream patch release on an ALREADY-pinned tag quietly changing a +# struct layout (this shouldn't happen for a fixed tag, since tags don't +# move -- but this is also a general regression test of the mirrors +# against reality, independent of whether anything upstream changed). +# +# Opens an issue, not a PR: unexpected drift here is surprising enough that +# it deserves a human looking at the actual cause before anything is +# auto-merged, rather than a bot quietly "fixing" it. + +on: + schedule: + - cron: '0 6 1 * *' # 1st of the month, 06:00 UTC + workflow_dispatch: + +jobs: + dump: + uses: ./.github/workflows/dump_structs.yml + + check: + needs: dump + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + - run: sudo apt-get update && sudo apt-get install -y gawk jq + - uses: actions/download-artifact@v4 + with: + path: dumps + - name: compare + run: | + set -euo pipefail + mapfile -t phpvs < <(tools/list_pinned_tags.sh) + # Positional: must match phpspy's version dispatch order, same as + # struct_dump.sh's own comment says. + nns=(70 71 72 73 74 80 81 82 83 84 85 86) + + : > drift.log + for i in "${!phpvs[@]}"; do + tag=${phpvs[$i]} + nn=${nns[$i]} + for pair in "ubuntu-24.04:x86_64" "ubuntu-24.04-arm:aarch64"; do + runs_on=${pair%%:*} + arch=${pair##*:} + raw="dumps/struct_dump-${runs_on}-${tag}/struct_dump.${tag}.out" + if [ ! -f "$raw" ]; then + echo "$nn/$arch ($tag): dump artifact missing entirely -- build likely failed" >> drift.log + continue + fi + before=$(sha256sum "structs/$arch/php_structs_$nn.h" | cut -d' ' -f1) + if tools/render_struct_update.sh "$nn" "$arch" "$raw" . 2>>drift.log; then + status=ok + else + status=needs_review + fi + after=$(sha256sum "structs/$arch/php_structs_$nn.h" | cut -d' ' -f1) + if [ "$before" != "$after" ] || [ "$status" = needs_review ]; then + echo "$nn/$arch ($tag): DRIFT ($status)" >> drift.log + fi + done + done + # This job only checks; it never commits. Revert any in-place + # edits render_struct_update.sh made so the working tree stays + # clean for the drift.log to be the sole output. + git checkout -- structs/ + cat drift.log + echo "drift_found=$([ -s drift.log ] && echo true || echo false)" >> "$GITHUB_OUTPUT" + id: compare + - name: open issue on drift + if: steps.compare.outputs.drift_found == 'true' + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + gh issue create \ + --title "struct_drift_check: unexpected drift in already-pinned PHP versions" \ + --body "$(printf 'Monthly drift check found the checked-in struct mirrors no longer match a fresh from-source build of an already-pinned PHP tag.\n\n```\n%s\n```\n\nSee the workflow run for full logs: %s/%s/actions/runs/%s' "$(cat drift.log)" "$GITHUB_SERVER_URL" "$GITHUB_REPOSITORY" "$GITHUB_RUN_ID")" diff --git a/struct_dump.sh b/struct_dump.sh index a66afc8..9443aba 100755 --- a/struct_dump.sh +++ b/struct_dump.sh @@ -11,6 +11,12 @@ fi # All PHP versions phpspy ships struct mirrors for. 8.6 has no release tag yet # (still dev), so it stays pinned to `master`, same as when the 8.6 structs # were first cut. +# +# Order matters: this must stay in the same order as phpspy's internal +# version dispatch (70, 71, 72, 73, 74, 80, 81, 82, 83, 84, 85, 86) -- +# .github/workflows/struct_drift_check.yml maps entries here to structs/ +# header files positionally, by that order, not by parsing version numbers +# out of the tag names. all_phpvs=( php-7.0.33 php-7.1.33 diff --git a/tools/find_php_updates.sh b/tools/find_php_updates.sh new file mode 100755 index 0000000..0fa300c --- /dev/null +++ b/tools/find_php_updates.sh @@ -0,0 +1,51 @@ +#!/bin/bash +# Compare the PHP release tags pinned in struct_dump.sh against what's +# actually tagged upstream, and report any series that has moved. +# +# Prints one "NN TAG" line per series that needs a fresh dump (NN = the +# phpspy-internal two-digit version, TAG = the newer php-src tag). Prints +# nothing if everything is current. On stderr, separately notes any PHP +# major.minor series that exists upstream but isn't tracked by phpspy at +# all (e.g. a future 8.7) -- that needs a human, not this script, since +# supporting a new series is more than a struct-offset refresh. +# +# Usage: tools/find_php_updates.sh [path-to-struct_dump.sh] + +set -euo pipefail + +this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) +dump_sh="${1:-$this_dir/../struct_dump.sh}" + +declare -A series=( + [70]=7.0 [71]=7.1 [72]=7.2 [73]=7.3 [74]=7.4 + [80]=8.0 [81]=8.1 [82]=8.2 [83]=8.3 [84]=8.4 [85]=8.5 +) + +mapfile -t all_tags < <( + git ls-remote --tags https://github.com/php/php-src.git 'refs/tags/php-*' \ + | sed 's#.*refs/tags/##' \ + | grep -E '^php-[0-9]+\.[0-9]+\.[0-9]+$' +) + +for nn in "${!series[@]}"; do + prefix="php-${series[$nn]}." + latest=$(printf '%s\n' "${all_tags[@]}" | grep -F "$prefix" | sort -V | tail -1) + [ -z "$latest" ] && continue + pinned=$(grep -oE "php-${series[$nn]//./\\.}\.[0-9]+" "$dump_sh" | head -1) + if [ "$latest" != "$pinned" ]; then + echo "$nn $latest" + fi +done + +# 8.6 is pinned to `master` (no release tag yet) -- flag it the moment one exists. +php86tag=$(printf '%s\n' "${all_tags[@]}" | grep -E '^php-8\.6\.' | sort -V | tail -1) +if [ -n "$php86tag" ]; then + echo "86 $php86tag" +fi + +# Informational only: a series beyond what phpspy tracks at all. +newest_series=$(printf '%s\n' "${all_tags[@]}" | grep -oE '^php-[0-9]+\.[0-9]+' | sed 's/^php-//' | sort -V -u | tail -1) +known_max="8.6" +if [ "$(printf '%s\n%s\n' "$known_max" "$newest_series" | sort -V | tail -1)" != "$known_max" ]; then + echo "note: upstream has a newer series ($newest_series) than phpspy tracks (up to $known_max) -- this needs a human to add support, not an automated struct refresh" >&2 +fi diff --git a/tools/list_pinned_tags.sh b/tools/list_pinned_tags.sh new file mode 100755 index 0000000..08bcc3e --- /dev/null +++ b/tools/list_pinned_tags.sh @@ -0,0 +1,18 @@ +#!/bin/bash +# Print struct_dump.sh's all_phpvs array, one tag per line. Single source +# of truth for "every version phpspy currently ships struct mirrors for" -- +# both struct_dump.sh itself and .github/workflows/dump_structs.yml's matrix +# read from here rather than keeping their own separate copies of the list. +# +# Usage: tools/list_pinned_tags.sh [struct_dump.sh path] + +set -euo pipefail + +this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) +dump_sh="${1:-$this_dir/../struct_dump.sh}" + +awk ' + /^all_phpvs=\(/ { in_arr = 1; next } + in_arr && /^\)/ { in_arr = 0; next } + in_arr { gsub(/^[ \t]+|[ \t]+$/, ""); if (length($0)) print } +' "$dump_sh" diff --git a/tools/parse_struct_dump.awk b/tools/parse_struct_dump.awk new file mode 100644 index 0000000..f1baa9d --- /dev/null +++ b/tools/parse_struct_dump.awk @@ -0,0 +1,25 @@ +# Normalize struct_dump.gdb output into flat "Type.field offset" lines. +# +# Usage: awk -f parse_struct_dump.awk struct_dump.SOMEVERSION.out +# +# Input looks like: +# zend_array +# type = struct _zend_array +# type = uint32_t +# nTableMask 12 +4 +# ... +# +# Output: one line per named field, "Type.field offset", sorted by nothing +# in particular (caller's problem). Anonymous union members (u1, u2, +# common) are dropped -- they're structural, not addressable fields, and +# phpspy_trace.c never reads them directly. + +/^[a-zA-Z_]+$/ { s = $1; next } +/type =/ { next } +/^ [A-Za-z_.]+ [0-9]+ \+[0-9]+$/ { + f = $1 + n = split(f, p, ".") + f = p[n] + if (f == "u1" || f == "u2" || f == "common") next + print s "." f, $2 +} diff --git a/tools/patch_struct_header.awk b/tools/patch_struct_header.awk new file mode 100644 index 0000000..9dacbd1 --- /dev/null +++ b/tools/patch_struct_header.awk @@ -0,0 +1,151 @@ +# Patch an existing structs/{arch}/php_structs_NN.h mirror header's field +# offsets (and the padN[] sizes that precede them) to match a fresh +# struct_dump.gdb probe, WITHOUT re-deriving struct layout from scratch. +# +# Scope, deliberately narrow: this only fixes the shape of drift actually +# observed across every PHP version and both architectures so far -- a +# single named field's absolute offset moving, with an immediately +# preceding `uint8_t padN[...]` line able to absorb the delta. Everything +# outside that shape is left untouched and reported on stderr for a human +# to look at; this script never invents new padding layouts, never adds or +# removes fields, and never touches union blocks (zend_function, the zval +# u1/u2 members) at all -- those are flagged only. +# +# Why this is safe without tracking a running cursor: every field's offset +# comment in these headers is an ABSOLUTE byte offset from the start of its +# own struct, not a cumulative sum of preceding fields. So each field can be +# checked and patched independently against the fresh dump, in isolation, +# regardless of what else in the struct did or didn't change. +# +# Line format (verified against the checked-in headers): 4-space indent, +# then the C type left-justified in a 24-char field, then the +# name+decorators+semicolon left-justified in a 24-char field, then +# "/* " + offset left-justified in a 9-char field + "+" + size + " */". +# Reconstructing lines with this exact rule (rather than in-place text +# substitution) is what keeps column alignment correct when a number's +# digit count changes. +# +# Usage: +# gawk -f patch_struct_header.awk -v dumpfile=OFFSETS.txt HEADER.h > HEADER.h.new +# +# OFFSETS.txt is the output of parse_struct_dump.awk: lines of +# "Type.field offset". +# +# Exit status is nonzero if anything needed human review (see stderr). + +BEGIN { + if (dumpfile == "") { + print "patch_struct_header.awk: -v dumpfile=... is required" > "/dev/stderr" + exit 2 + } + while ((getline dline < dumpfile) > 0) { + split(dline, dparts, " ") + dump_off[dparts[1]] = dparts[2] + } + close(dumpfile) + + in_block = 0 + needs_review = 0 +} + +match($0, /^(struct|union) __attribute__\(\(__packed__\)\) _([A-Za-z_]+)_([0-9]+) \{$/, m) { + in_block = 1 + is_union = (m[1] == "union") + type_key = m[2] + nblock = 0 + print + next +} + +in_block && /^\};$/ { + process_block() + in_block = 0 + print + next +} + +in_block { + nblock++ + line[nblock] = $0 + typecol[nblock] = substr($0, 5, 24) + namecol[nblock] = substr($0, 29, 24) + + if (match($0, /^ uint8_t {17}pad[0-9]+\[([0-9]+)\]; *\/\* *([0-9]+) *\+([0-9]+) \*\/$/, pm)) { + kind[nblock] = "pad" + pad_size[nblock] = pm[1] + 0 + off[nblock] = pm[2] + 0 + next + } + + if (match($0, /\/\* *([0-9]+) *\+([0-9]+) \*\/$/, fm)) { + # Named field. Extract NAME from namecol: strip a leading `*`, + # trailing `;` and any `[N]` array suffix. + nm = namecol[nblock] + gsub(/^ */, "", nm) + sub(/^\*/, "", nm) + sub(/\[[0-9]+\]/, "", nm) + sub(/;.*$/, "", nm) + kind[nblock] = "field" + fname[nblock] = nm + off[nblock] = fm[1] + 0 + size[nblock] = fm[2] + 0 + next + } + + kind[nblock] = "other" + next +} + +# Anything not inside a block (typedefs, blank lines, #ifndef/#endif, ...) +# is passed through completely unchanged. +{ print } + +END { + if (needs_review) exit 1 +} + +function render(type_text, name_text, o, s) { + return sprintf(" %-24s%-24s/* %-9d+%d */", type_text, name_text, o, s) +} + +function process_block( i, key, want, delta, newpad, prev_is_pad, nametext) { + for (i = 1; i <= nblock; i++) { + if (kind[i] != "field") continue + key = type_key "." fname[i] + if (!(key in dump_off)) continue + want = dump_off[key] + 0 + if (want == off[i]) continue + + if (is_union) { + printf("patch_struct_header: %s union field %s offset differs (header=%d dump=%d) -- union blocks are never auto-patched, review by hand\n", type_key, fname[i], off[i], want) > "/dev/stderr" + needs_review = 1 + continue + } + + prev_is_pad = (i > 1 && kind[i-1] == "pad") + if (!prev_is_pad) { + printf("patch_struct_header: %s.%s offset differs (header=%d dump=%d) but has no immediately preceding pad to absorb it -- review by hand\n", type_key, fname[i], off[i], want) > "/dev/stderr" + needs_review = 1 + continue + } + + delta = want - off[i] + newpad = pad_size[i-1] + delta + if (newpad < 0) { + printf("patch_struct_header: %s.%s would need a negative pad (header=%d dump=%d, preceding pad=%d) -- review by hand\n", type_key, fname[i], off[i], want, pad_size[i-1]) > "/dev/stderr" + needs_review = 1 + continue + } + + nametext = namecol[i-1] + sub(/\[[0-9]+\]/, "[" newpad "]", nametext) + line[i-1] = render(typecol[i-1], nametext, off[i-1], newpad) + pad_size[i-1] = newpad + + line[i] = render(typecol[i], namecol[i], want, size[i]) + + printf("patch_struct_header: %s.%s %d -> %d (preceding pad %d -> %d)\n", type_key, fname[i], off[i], want, pad_size[i-1] - delta, pad_size[i-1]) > "/dev/stderr" + off[i] = want + } + for (i = 1; i <= nblock; i++) print line[i] +} diff --git a/tools/render_struct_update.sh b/tools/render_struct_update.sh new file mode 100755 index 0000000..0bc660e --- /dev/null +++ b/tools/render_struct_update.sh @@ -0,0 +1,43 @@ +#!/bin/bash +# Given a fresh struct_dump.gdb output for one PHP version on one arch, +# patch the corresponding checked-in mirror header in place. +# +# Usage: tools/render_struct_update.sh [repo_root] +# +# Exit status: 0 = header already matched or was patched cleanly. +# 1 = patch_struct_header.awk flagged something for human +# review (see stderr); the header is left untouched. + +set -euo pipefail + +this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) +nn=$1 +arch=$2 +raw_dump=$3 +repo_root=${4:-$this_dir/..} + +header="$repo_root/structs/$arch/php_structs_$nn.h" +if [ ! -f "$header" ]; then + echo "render_struct_update: no such header: $header" >&2 + exit 2 +fi + +offsets=$(mktemp) +trap 'rm -f "$offsets"' EXIT +gawk -f "$this_dir/parse_struct_dump.awk" "$raw_dump" > "$offsets" + +patched=$(mktemp) +if gawk -f "$this_dir/patch_struct_header.awk" -v dumpfile="$offsets" "$header" > "$patched" 2>"$patched.log"; then + rv=0 +else + rv=1 +fi +cat "$patched.log" >&2 + +if ! diff -q "$patched" "$header" >/dev/null 2>&1; then + cp "$patched" "$header" + echo "render_struct_update: updated $header" >&2 +fi +rm -f "$patched" "$patched.log" + +exit "$rv" diff --git a/tools/update_pinned_tag.sh b/tools/update_pinned_tag.sh new file mode 100755 index 0000000..99dfc90 --- /dev/null +++ b/tools/update_pinned_tag.sh @@ -0,0 +1,30 @@ +#!/bin/bash +# Update struct_dump.sh's pinned tag for whichever series belongs +# to (e.g. "php-8.1.34" replaces whatever "php-8.1.NNN" line is currently +# pinned). PHP 8.6 is a special case: it's pinned to the literal `master` +# until a real release tag exists, so a `php-8.6.*` tag replaces that +# literal line instead of a numbered one. +# +# Usage: tools/update_pinned_tag.sh [struct_dump.sh path] + +set -euo pipefail + +new_tag=$1 +this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) +dump_sh="${2:-$this_dir/../struct_dump.sh}" + +if [[ "$new_tag" =~ ^php-8\.6\. ]]; then + if ! grep -qE '^ *master *$' "$dump_sh"; then + echo "update_pinned_tag: no literal 'master' line found for 8.6" >&2 + exit 1 + fi + sed -i -E "s/^( *)master( *)$/\1${new_tag}\2/" "$dump_sh" +else + series=$(echo "$new_tag" | sed -E 's/\.[0-9]+$//') + series_escaped=$(printf '%s' "$series" | sed -E 's/[.[\*^$]/\\&/g') + if ! grep -qE "^ *${series_escaped}\.[0-9]+ *$" "$dump_sh"; then + echo "update_pinned_tag: no existing ${series}.NNN line found to replace" >&2 + exit 1 + fi + sed -i -E "s/^( *)${series_escaped}\.[0-9]+( *)$/\1${new_tag}\2/" "$dump_sh" +fi From b817df31f31f7f5ac0b0fbd06523576c9efad03b Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 08:36:23 -0400 Subject: [PATCH 5/9] Clear the inherited github.com auth header before hitting an unrelated repo actions/checkout@v4 (with the default persist-credentials: true) sets http.https://github.com/.extraheader in the checked-out repo's local git config, scoped to that repo's own token. Since git matches extraheader config by URL prefix rather than by remote name, any git command run with CWD inside that checkout -- including this script's `git ls-remote` against an unrelated public repo (php/php-src) -- inherits that header too, and GitHub rejects it outright ("Invalid username or token"), failing in ~0.3s rather than actually attempting the read. Reproduced locally by setting a bogus extraheader and confirming `git ls-remote` against php-src fails the same way in the same ~0.3s; `-c http.https://github.com/.extraheader=` clears it for just this invocation and resolves it. This is also what caused check_struct_versions.yml's first real test run to fail immediately in CI (https://github.com/rlerdorf/phpspy/actions/runs/34226820940). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- tools/find_php_updates.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tools/find_php_updates.sh b/tools/find_php_updates.sh index 0fa300c..7952dfd 100755 --- a/tools/find_php_updates.sh +++ b/tools/find_php_updates.sh @@ -22,7 +22,13 @@ declare -A series=( ) mapfile -t all_tags < <( - git ls-remote --tags https://github.com/php/php-src.git 'refs/tags/php-*' \ + # -c ...extraheader= clears any Authorization header a caller's git + # config may have set for github.com URLs in general (e.g. a CI + # checkout step's persisted credentials, scoped to a *different* repo) + # -- inherited into a request for an unrelated public repo like this + # one, such a header gets rejected outright rather than ignored. + git -c http.https://github.com/.extraheader= \ + ls-remote --tags https://github.com/php/php-src.git 'refs/tags/php-*' \ | sed 's#.*refs/tags/##' \ | grep -E '^php-[0-9]+\.[0-9]+\.[0-9]+$' ) From fb5e730c37cd29bdabda2122d8f7382746deabb6 Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 08:37:52 -0400 Subject: [PATCH 6/9] debug: add set -x to diagnose CI failure (will revert) Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- tools/find_php_updates.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/find_php_updates.sh b/tools/find_php_updates.sh index 7952dfd..1bcc975 100755 --- a/tools/find_php_updates.sh +++ b/tools/find_php_updates.sh @@ -11,7 +11,7 @@ # # Usage: tools/find_php_updates.sh [path-to-struct_dump.sh] -set -euo pipefail +set -euxo pipefail this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) dump_sh="${1:-$this_dir/../struct_dump.sh}" From bfad410cc5b98c47ec49f6c2b258500a1013973b Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 08:39:54 -0400 Subject: [PATCH 7/9] Fix find_php_updates.sh: a legitimate zero-match grep must not abort the script Real bug, not just a debug leftover: with `set -e -o pipefail`, each `x=\$(... | grep ... | ...)` assignment aborts the whole script the moment grep finds zero matches, even though "nothing matched" is often the correct, expected result here -- e.g. there is currently no php-8.6.* release tag at all, and that's not a failure, just nothing to report. Caught by actually running this in CI (the previous debug commit's `set -x` trace), which is what exposed that my own earlier local testing sessions were fooled: wrapping the script in `time ...` was masking its real exit code, so "looks fine locally" was never a valid signal for this bug. Direct invocation without the `time` wrapper now correctly shows the failure before this fix and success after. Also reverts the prior commit's temporary `set -x` debug tracing. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- tools/find_php_updates.sh | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/tools/find_php_updates.sh b/tools/find_php_updates.sh index 1bcc975..1cad885 100755 --- a/tools/find_php_updates.sh +++ b/tools/find_php_updates.sh @@ -11,7 +11,13 @@ # # Usage: tools/find_php_updates.sh [path-to-struct_dump.sh] -set -euxo pipefail +set -euo pipefail + +# NOTE: several lines below intentionally end in `|| true`. Under +# pipefail, a `grep` that legitimately finds nothing (e.g. "no php-8.6.* +# release tag exists yet") exits nonzero, and set -e would otherwise treat +# that as a script failure rather than the valid "nothing found" result it +# actually is. this_dir=$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null && pwd) dump_sh="${1:-$this_dir/../struct_dump.sh}" @@ -35,22 +41,22 @@ mapfile -t all_tags < <( for nn in "${!series[@]}"; do prefix="php-${series[$nn]}." - latest=$(printf '%s\n' "${all_tags[@]}" | grep -F "$prefix" | sort -V | tail -1) + latest=$(printf '%s\n' "${all_tags[@]}" | grep -F "$prefix" | sort -V | tail -1) || true [ -z "$latest" ] && continue - pinned=$(grep -oE "php-${series[$nn]//./\\.}\.[0-9]+" "$dump_sh" | head -1) + pinned=$(grep -oE "php-${series[$nn]//./\\.}\.[0-9]+" "$dump_sh" | head -1) || true if [ "$latest" != "$pinned" ]; then echo "$nn $latest" fi done # 8.6 is pinned to `master` (no release tag yet) -- flag it the moment one exists. -php86tag=$(printf '%s\n' "${all_tags[@]}" | grep -E '^php-8\.6\.' | sort -V | tail -1) +php86tag=$(printf '%s\n' "${all_tags[@]}" | grep -E '^php-8\.6\.' | sort -V | tail -1) || true if [ -n "$php86tag" ]; then echo "86 $php86tag" fi # Informational only: a series beyond what phpspy tracks at all. -newest_series=$(printf '%s\n' "${all_tags[@]}" | grep -oE '^php-[0-9]+\.[0-9]+' | sed 's/^php-//' | sort -V -u | tail -1) +newest_series=$(printf '%s\n' "${all_tags[@]}" | grep -oE '^php-[0-9]+\.[0-9]+' | sed 's/^php-//' | sort -V -u | tail -1) || true known_max="8.6" if [ "$(printf '%s\n%s\n' "$known_max" "$newest_series" | sort -V | tail -1)" != "$known_max" ]; then echo "note: upstream has a newer series ($newest_series) than phpspy tracks (up to $known_max) -- this needs a human to add support, not an automated struct refresh" >&2 From 54fe9b9beda898aa930a87a07f78d67f872f0a6d Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 08:48:55 -0400 Subject: [PATCH 8/9] Grant the render job write permissions to push and open its PR The default GITHUB_TOKEN is read-only unless a job explicitly requests more, which the render job needs for both the git push and gh pr create at the end. Confirmed everything upstream of this works correctly first: detect found 3 real tag updates (php-8.1.34/8.2.33/8.3.33), all 6 dump cells succeeded, and render correctly determined these patch releases don't change any struct layout -- only struct_dump.sh's pinned tags needed updating. Only the final push/PR step failed, with exactly the 403 this fixes (https://github.com/rlerdorf/phpspy/actions/runs/34227369760). Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- .github/workflows/check_struct_versions.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/check_struct_versions.yml b/.github/workflows/check_struct_versions.yml index 8572ec1..a0fb4e2 100644 --- a/.github/workflows/check_struct_versions.yml +++ b/.github/workflows/check_struct_versions.yml @@ -76,6 +76,9 @@ jobs: render: needs: [detect, dump] runs-on: ubuntu-24.04 + permissions: + contents: write + pull-requests: write steps: - uses: actions/checkout@v4 - name: install gawk From 8fe92b43ec3ddb4e94b8135e320d9630ce063b05 Mon Sep 17 00:00:00 2001 From: Rasmus Lerdorf Date: Tue, 8 Sep 2026 09:17:51 -0400 Subject: [PATCH 9/9] Grant the check job write permissions for gh issue create Same class of bug as the render job's earlier 403: the default GITHUB_TOKEN is read-only unless a job explicitly requests more, and this job's final step (gh issue create on drift) needs issues: write. Caught by inspection while validating the workflow end-to-end -- this fork has Issues disabled entirely, so the real run found zero drift (correctly) and never reached that step, meaning the missing permission would have failed silently on the next repo where it actually triggers. Not yet verified against a real "drift found" run. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01V8kSnhaw7WUaR61G6cBp1w --- .github/workflows/struct_drift_check.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/struct_drift_check.yml b/.github/workflows/struct_drift_check.yml index 208727b..f738121 100644 --- a/.github/workflows/struct_drift_check.yml +++ b/.github/workflows/struct_drift_check.yml @@ -26,6 +26,8 @@ jobs: check: needs: dump runs-on: ubuntu-24.04 + permissions: + issues: write steps: - uses: actions/checkout@v4 - run: sudo apt-get update && sudo apt-get install -y gawk jq