model-auditing

AI Red Team Operations Console

A structured audit framework for ML pipelines. 14 stages, 100+ checks, and every leakage pattern that silent model failures are made of.

Public credit-risk models report 0.90+ AUC using post-loan leakage. This one strips it and reports an honest 0.73 — then runs a ReAct agent to audit itself.

Steering vectors with receipts: make one, catch one, deploy a calibrated one. pip install hidden-directions

Autonomous Update Review Architecture for Federated Learning. A full-stack platform for inspecting, scoring, and auditing model updates using SHAP explainability, anomaly detection, and ledger-based tracking. Built with FastAPI, React 19, and Flower.

Ad-funded LLMs as the default route to commercial secret loyalties — Secret Loyalties Hackathon paper + black-box audit harness.

Layer-wise representation-drift screening for related open-weight language-model checkpoints

Third-party-verifiable model-authenticity audit for the javis.bot Claude/GPT relay — open-source probe, runs on GitHub Actions, code/runs/results all public.

Shift type, not magnitude, determines ML failure modes under deployment shift — a cross-domain audit protocol and benchmark

Open-source toolkit for Fourier-based audits of trained quantum neural networks.

A method-neutral protocol for recording and verifying evidence that latent model states causally influenced downstream decisions, outputs, or actions, without requiring storage of raw internal activations.

A production-grade LLM Evaluation & Benchmarking Framework for systematic model auditing. Features parallel benchmarking, fairness/bias detection, MMLU integration, and a real-time analytics dashboard powered by React and FastAPI.

Production-grade platform for auditing AI/ML models for fairness, explainability, and robustness. JWT auth, SHA-256 model verification, paginated REST API, dark web dashboard. 30/30 tests passing.

Reproducible pipeline for silent-failure auditing in ECG accept-sets (MIT-BIH) with Newton–Puiseux onset scoring

Workspace-lens audit cards and training-workflow scaffolds for open-weight language models.

Интерпретируемый аудит black-box моделей временных рядов с ограниченным бюджетом контрсвидетельств, адаптивной остановкой и оценкой применимости на edge-устройствах.

Subgroup-stratified, calibration-aware fairness auditing for ML models: DeLong AUC confidence intervals, per-subgroup calibration error, multiple-comparison-corrected significance, and a novel five-axis cross-platform protocol (CPFE). Grounded in peer-reviewed methods.

SHAP-based bias detection and model interpretability auditing framework. Feature importance analysis and ML audit automation.

studying model misalignment via spectral analysis

Audit a public AI text classifier for bias and explain its decisions. Counterfactual fairness probing (Equity Evaluation Corpus style) + occlusion-based token attribution, on a black-box model using only inputs and outputs. Python · Transformers · PyTorch.