Add the paper's benchmark artifact and PDF

README.md

@@ -300,6 +300,34 @@ python training/train_retriever.py \
 Do not publish the demo-derived checkpoint as a research result. Build workflow-disjoint train,
 calibration, and hidden test splits first.
 
+## Paper experiments
+
+The CLINC150 and BANKING77 results reported in the FlowRoute paper are
+reproduced by `experiments/run_benchmark.py`. The script downloads both
+datasets from their official repositories, verifies their SHA-256 digests,
+pins the `all-MiniLM-L6-v2` model revision, keeps the test splits out of every
+tuning decision, and records complete per-seed results.
+
+```bash
+cd experiments
+python -m venv .venv
+.venv/bin/pip install --index-url https://download.pytorch.org/whl/cpu torch==2.8.0
+.venv/bin/pip install -r requirements.txt
+HF_HOME=cache .venv/bin/python run_benchmark.py
+```
+
+`experiments/results/benchmark_results.json` is the record the paper's tables
+are derived from: per-seed metrics, selected thresholds, model revision, source
+URLs, checksums, and the environment. `experiments/results/ranking_summary.csv`
+is a compact view of the ranking table.
+
+Note that the scoring rule benchmarked in the paper is not the one used by
+`flowroute.backends.HuggingFaceRetriever`. The paper embeds each workflow
+description and each approved example separately and takes the maximum example
+similarity; the shipped retriever embeds one concatenated capability string per
+contract. The two are not interchangeable, and the paper's numbers correspond
+to the former.
+
 ## Repository map
 
 ```text

experiments/README.md

@@ -0,0 +1,37 @@
+# FlowRoute paper benchmark
+
+This directory reproduces every empirical number in the FlowRoute paper. The
+benchmark downloads the official CLINC150 and BANKING77 files, verifies their
+SHA-256 digests, and evaluates the pinned `all-MiniLM-L6-v2` model revision.
+
+The public test splits are never used to select the description/example mixing
+weight or abstention thresholds. CLINC150 uses its official validation split
+and combines its official out-of-scope training and validation sets for
+threshold selection. Both calibration constraints use the upper endpoints of
+two-sided 95% Wilson intervals rather than point estimates.
+Because BANKING77 has no official validation split, the script deterministically
+reserves 20 training examples per intent before selecting workflow examples.
+
+## Run
+
+Python 3.12 was used for the paper. Create an environment and install the pinned
+dependencies:
+
+```bash
+python -m venv .venv
+.venv/bin/pip install --index-url https://download.pytorch.org/whl/cpu torch==2.8.0
+.venv/bin/pip install -r requirements.txt
+HF_HOME=cache .venv/bin/python run_benchmark.py
+```
+
+The script writes:
+
+- `results/benchmark_results.json`: complete per-seed metrics, thresholds,
+  environment, model revision, source URLs, and checksums.
+- `results/ranking_summary.csv`: compact ranking table.
+- `environment-lock.txt`: the complete package-version record from the
+  reported run. It is an environment record, not a cross-platform installer.
+
+Downloaded dataset files and model cache files are intentionally excluded from
+the source bundle. The original licenses apply: CC BY 3.0 for CLINC150, CC BY
+4.0 for BANKING77, and Apache 2.0 for `all-MiniLM-L6-v2`.

experiments/environment-lock.txt

@@ -0,0 +1,34 @@
+# Exact `pip freeze` record from the reported Linux x86_64 benchmark run.
+Jinja2==3.1.6
+MarkupSafe==3.0.3
+PyYAML==6.0.3
+certifi==2026.7.22
+charset-normalizer==3.5.1
+cloudpickle==3.1.2
+filelock==3.32.3
+fsspec==2026.7.0
+hf-xet==1.6.0
+huggingface_hub==0.36.2
+idna==3.19
+joblib==1.6.0
+mpmath==1.3.0
+narwhals==2.26.0
+networkx==3.6.1
+numpy==2.5.3
+packaging==26.3
+pillow==12.3.0
+regex==2026.9.10
+requests==2.34.2
+safetensors==0.8.0
+scikit-learn==1.9.1
+scipy==1.18.1
+sentence-transformers==5.1.2
+setuptools==78.1.0
+sympy==1.14.0
+threadpoolctl==3.6.0
+tokenizers==0.22.2
+torch==2.8.0+cpu
+tqdm==4.70.1
+transformers==4.57.6
+typing_extensions==4.16.0
+urllib3==2.7.0

experiments/requirements.txt

@@ -0,0 +1,4 @@
+numpy==2.5.3
+scikit-learn==1.9.1
+sentence-transformers==5.1.2
+torch==2.8.0