data-contamination

The Paper List on Data Contamination for Large Language Models Evaluation.

This python code performs an efficient speech reverberation starting from a dataset of close-talking speech signals and a collection of acoustic impulse responses.

Python .pyc decompiler (3.0–3.14) with a contamination-aware benchmark harness. Rule-only pass + one Codex call per module; evaluated on fuzz-synthetic (LLM-naïve) and *-obf (anonymised) corpora to put a number on the memorisation share. Three independent PyPI packages: pychd, pychd-pyfuzz, pychd-pyobf.

The official implementation of the paper "Data Contamination Calibration for Black-box LLMs" (ACL 2024)

The official repository for the paper entitled "Time Travel in LLMs: Tracing Data Contamination in Large Language Models."

Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.

DICE: Detecting In-distribution Data Contamination with LLM's Internal State

The official repository for the paper entitled "Data Contamination Quiz: A Tool to Detect and Estimate Contamination in Large Language Models."

Point-in-time correct LLM instrumentation — tracing, version pinning and look-ahead-bias protection for research pipelines. pip install traceguard

Zero-dependency CLI + library to detect benchmark/eval data contamination against a training corpus via n-gram overlap, flag leaks, and emit a cleaned dataset.

Zero-CoT Probe (ZCP) for detecting evasive data contamination in LLMs. Paper Link: http://arxiv.org/abs/2605.21856

Time-aware dataset forensics and early-warning risk prediction for LLM fine-tuning. Predicts whether a dataset will damage a model — before the fine-tuning damage becomes visible.

Scan evaluation data for n-gram overlap, canaries, answer patterns, hashes, and optional embedding similarity.

Leakage-free real-time evaluation of open-weights LLMs for US CPI inflation forecasting. Introduces the memorization premium (seen vs. unseen forecast-error gap) and a three-role decomposition (direct forecaster, FOMC-text extractor, combiner). Reproduces every number in the IJF manuscript's Table 3 from the committed checkpoint.

ContamCheckr: a local, CPU-friendly auditor for benchmark contamination evidence in causal language models.

Turn code repos and curated visual repos into contamination-checked training datasets for local fine-tuning.

A contamination-resistant research platform for testing whether public market data carries tradeable information. Ten pre-registered experiments on an explore -> holdout -> live-forward design; none survived.

Deterministic evaluation harness for AP document-matching agents. Scores 3-way findings against a hand-audited, held-out golden dataset: per-category precision and recall, over-flagging measured on a zero-defect control, byte-reproducible scorecards, answer key structurally out of reach.

Scores how much of an eval item already appears inside a training corpus, asymmetrically

Toolkit for detecting benchmark/eval-data contamination in LLM training corpora