feat: add reflection probing and steering pipelines - #2

Merged

xzAscC merged 43 commits into

mainfrom

Jul 16, 2026

Commits on Jul 16, 2026

chore(deps): add transformers, torch, datasets for inference
xzAscC
committed
ddf5e10
test(inference): add TDD tests for inference module
xzAscC
committed
9124e03
feat(inference): implement MATH500 inference with Qwen3.5-0.8B
xzAscC
committed
50db867
feat(inference): implement MATH500 inference with Qwen3.5-0.8B
xzAscC
committed
6769b10
feat(inference): implement MATH500 inference with Qwen3.5-0.8B
xzAscC
committed
c699d23
fix(inference): replace Any type with proper types, format tests
xzAscC
committed
0e95c2b
feat(eval): add LLM-as-Judge evaluation module
xzAscC
committed
2f83496
feat: add reflection token diagnosis tool
xzAscC
committed
e023022
feat(types): add ExtractVectorsConfig and SteeringVectorResult for steering vectors
xzAscC
committed
3f31e4f
feat(steering): implement classify_samples, find_reflection_token_position, extract_activation_at_position, compute_difference_in_means, save_steering_vectors

xzAscC
and
Sisyphus
committed
6228103
feat(steering): implement extract_batch_activations with batch processing

xzAscC
and
Sisyphus
committed
f9b5bd7
feat(steering): implement classify_samples, find_reflection_token_position, extract_activation_at_position, extract_batch_activations, compute_difference_in_means, save_steering_vectors

xzAscC
and
Sisyphus
committed
d39b410
feat(steering): implement extract_steering_vectors pipeline and extract-vectors CLI command

xzAscC
and
Sisyphus
committed
502b26e
feat(steering): add complete steering inference infrastructure
xzAscC
committed
512b774
fix(steering): dtype mismatch and dataset split handling
xzAscC
committed
2ff6637
feat(linear-probe): implement linear probe module for reflection token separability
xzAscC
committed
e4a0432
feat(linear-probe): add run_linear_probe pipeline and update exports
xzAscC
committed
2529c2e
chore: configure local research file ignores

xzAscC
and
sisyphus-dev-ai
committed
db27028
chore: exclude research notebooks from Ruff

xzAscC
and
sisyphus-dev-ai
committed
280ee8b
feat(utils): add shared model loading helpers

xzAscC
and
sisyphus-dev-ai
committed
ef01088
refactor(inference): share batch processing helpers

xzAscC
and
sisyphus-dev-ai
committed
437239d
feat(prompts): centralize evaluation prompt templates

xzAscC
and
sisyphus-dev-ai
committed
8923624
refactor(linear-probe): reuse shared model utilities

xzAscC
and
sisyphus-dev-ai
committed
f9b9ca7
refactor(steering): reuse shared model loading

xzAscC
and
sisyphus-dev-ai
committed
adb56ab
refactor(judges): consolidate LLM evaluation classes

xzAscC
and
sisyphus-dev-ai
committed
9eafb76
feat(roscoe): add reasoning quality diagnosis

xzAscC
and
sisyphus-dev-ai
committed
ac7486a
test(datasets): mock Hugging Face loading

xzAscC
and
sisyphus-dev-ai
committed
ccd7469
style(cli): simplify command help text

xzAscC
and
sisyphus-dev-ai
committed
a3d936c
feat(scripts): add cross-family judge tooling

xzAscC
and
sisyphus-dev-ai
committed
f17b6ec
chore(scripts): add experiment runner wrappers

xzAscC
and
sisyphus-dev-ai
committed
c75118e
build(deps): add Jupyter runtime

xzAscC
and
sisyphus-dev-ai
committed
f970b82
docs(readme): update paper citation

xzAscC
and
sisyphus-dev-ai
committed
4b73761
build(types): follow untyped dependency imports

xzAscC
and
sisyphus-dev-ai
committed
a3cbafe
refactor(models): type model lifecycle boundaries

xzAscC
and
sisyphus-dev-ai
committed
85b382e
fix(inference): enforce limits and decode generated tokens

xzAscC
and
sisyphus-dev-ai
committed
405c7c3
fix(judges): decode only newly generated tokens

xzAscC
and
sisyphus-dev-ai
committed
1984cf5
fix(steering): type hooks and decode generated tokens

xzAscC
and
sisyphus-dev-ai
committed
751a94a
fix(steering-vectors): avoid duplicate OOM activations

xzAscC
and
sisyphus-dev-ai
committed
849c961
refactor(datasets): validate untyped dataset rows

xzAscC
and
sisyphus-dev-ai
committed
2f75e3d
refactor(evaluation): normalize result metadata types

xzAscC
and
sisyphus-dev-ai
committed
adac210
refactor(linear-probe): type sklearn boundaries

xzAscC
and
sisyphus-dev-ai
committed
6bb9a3d
fix(scripts): prevent wrapper argument injection

xzAscC
and
sisyphus-dev-ai
committed
998a67e
docs(readme): document current research pipelines

xzAscC
and
sisyphus-dev-ai
committed
37b4170