feat: Implement artifact-backed API for transaction risk prediction - #1
Merged
Merged
Conversation
- Added a new Flask API in `src/api/app.py` to serve risk predictions based on transaction features. - Introduced health check endpoint to verify model availability. - Updated prediction endpoint to return risk probability and review requirements. - Refactored transaction processing in `src/api/simple_api_server.py` to use risk probability instead of risk score. - Modified real-time dashboard to display risk probability. - Created data loading functionality in `src/data/loader.py` for SAML-D dataset. - Added evaluation metrics and explainability functions in `src/evaluation/metrics.py` and `src/evaluation/explainability.py`. - Developed feature engineering functions for behavioral and transaction features in `src/features/behavioral_features.py` and `src/features/transaction_features.py`. - Implemented model training and calibration logic in `src/models/train.py` and `src/models/calibration.py`. - Added unit tests for feature engineering and inference in `tests/test_features.py` and `tests/test_inference.py`. - Updated utility scripts for system startup and transaction generation to reflect API changes.
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
It introduces a new end-to-end ML training/inference/API surface with identified security/performance concerns that should be validated and benchmarked in a full human review.
Pull request overview
This PR introduces an artifact-backed transaction risk modeling pipeline (feature engineering → training → calibration → saved artifact) and a new Flask API that serves calibrated risk probabilities from feature-store-supplied inputs, with supporting scripts, tests, and demo/UI updates.
Changes:
- Added artifact-backed inference helpers and a new Flask API (
/api/health,/api/model/info,/api/predict) that serves calibratedrisk_probability. - Implemented SAML-D data loading, feature engineering (transaction + behavioral), model training + calibration, and evaluation utilities, plus scripts to build features/train/run ablations.
- Updated the legacy demo server, ingestion utilities, and dashboard to surface
risk_probabilityand adjusted repo hygiene for datasets/artifacts (gitignore, LFS removal, docs).
File summaries
| File | Description |
|---|---|
| tests/test_leakage.py | Adds a regression check preventing identifier/target leakage into MODEL_FEATURES. |
| tests/test_inference.py | Adds unit tests for inference input validation and API health behavior when artifacts are missing. |
| tests/test_features.py | Adds unit coverage for derived transaction-only features without encoding account IDs. |
| src/utils/test_ingestion.py | Updates demo output to print risk_probability. |
| src/utils/start_system.py | Switches startup to the new artifact-backed API module; disables demo stream by default. |
| src/utils/simple_ingestion.py | Updates demo stream messaging/output to risk_probability and clarifies non-realism. |
| src/models/train.py | Introduces feature lists, preprocessing, temporal splitting, and model fitting utilities. |
| src/models/inference.py | Adds artifact-backed input validation + calibrated probability scoring utilities. |
| src/models/calibration.py | Adds a lightweight probability calibration wrapper (logit + logistic regression). |
| src/models/baseline.py | Adds a logistic baseline model builder for benchmarking. |
| src/models/init.py | Package marker for src.models. |
| src/features/transaction_features.py | Adds per-transaction feature derivations (time, amount, currency, cross-border). |
| src/features/behavioral_features.py | Adds historical behavioral/network features via DuckDB windows + additional Python aggregation. |
| src/features/init.py | Package marker for src.features. |
| src/evaluation/metrics.py | Adds alert-rate metrics and standard probabilistic evaluation metrics. |
| src/evaluation/explainability.py | Adds SHAP explainer helpers for model interpretation. |
| src/evaluation/init.py | Package marker for src.evaluation. |
| src/data/loader.py | Adds SAML-D loader with schema expectations, LFS-pointer detection, and timestamp creation. |
| src/data/init.py | Package marker for src.data. |
| src/dashboard/real_time_dashboard.html | Updates alert rendering to display risk_probability. |
| src/api/simple_api_server.py | Refactors legacy endpoint payloads to return risk_probability keys. |
| src/api/app.py | Adds the new artifact-backed Flask API (health, model/info, predict). |
| src/api/init.py | Adds module docstring for HTTP interface package. |
| scripts/train_model.py | Adds offline training script that produces a risk_model.joblib artifact. |
| scripts/feature_ablation.py | Adds a script to evaluate feature set ablations with alert-rate metrics. |
| scripts/build_features.py | Adds a script to build and persist the engineered feature dataset (parquet). |
| SAML-D.csv | Removes the repo-root CSV LFS pointer. |
| requirements.txt | Updates/modernizes Python dependencies for the new pipeline/API/tooling. |
| reports/figures/.gitkeep | Keeps reports figures directory in git. |
| reports/.gitkeep | Keeps reports directory in git. |
| notebooks/03_backtesting.ipynb | Adds a backtesting notebook scaffold. |
| notebooks/02_model_research.ipynb | Adds a (large) research notebook snapshot. |
| notebooks/01_eda.ipynb | Adds an EDA notebook snapshot. |
| data/README.md | Documents SAML-D dataset expectations and licensing/citation. |
| data/raw/.gitkeep | Keeps raw data directory in git while ignoring actual datasets. |
| artifacts/.gitkeep | Keeps artifacts directory in git while ignoring generated artifacts. |
| .gitignore | Refactors ignore rules for datasets, artifacts, notebooks, and test outputs. |
| .gitattributes | Removes Git LFS configuration for CSVs. |
| .firebaserc | Removes Firebase project config. |
Review details
Suppressed comments (1)
src/api/simple_api_server.py:349
- In bulk processing, the output key is
risk_probabilitybut the value is still carried in arisk_scorevariable. This naming mismatch makes it harder for readers to tell whether this is a calibrated probability or a legacy score.
results.append({
'transaction_id': tx_data['transaction_id'],
'risk_probability': risk_score,
'risk_level': risk_level,
'compliance_status': 'PENDING' if requires_review else 'APPROVED',
'requires_review': requires_review,
'flagged_features': flagged_features,
'processed_at': datetime.now().isoformat()
- Files reviewed: 27/39 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+166
to
+171
| sender_totals = {} | ||
| sender_squared_totals = {} | ||
| sender_receiver_totals = {} | ||
| concentration = [] | ||
|
|
||
| for _, timestamp_group in result.groupby("timestamp", sort=False): |
Comment on lines
+128
to
+136
| def build_preprocessor(feature_names=MODEL_FEATURES): | ||
| numeric_features = [ | ||
| feature for feature in feature_names | ||
| if feature in NUMERIC_FEATURES | ||
| ] | ||
| categorical_features = [ | ||
| feature for feature in feature_names | ||
| if feature in CATEGORICAL_FEATURES | ||
| ] |
Comment on lines
253
to
256
| return jsonify({ | ||
| 'transaction_id': data['transaction_id'], | ||
| 'risk_score': risk_score, | ||
| 'risk_probability': risk_score, | ||
| 'risk_level': risk_level, |
Comment on lines
+9
to
+30
| from datetime import datetime, timezone | ||
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| import joblib | ||
| from flask import Flask, jsonify, request | ||
| from flask_cors import CORS | ||
|
|
||
| from src.models.inference import ( | ||
| model_input_from_features, | ||
| predict_calibrated_probability, | ||
| probability_percentile, | ||
| ) | ||
|
|
||
| PROJECT_ROOT = Path(__file__).resolve().parents[2] | ||
| MODEL_PATH = PROJECT_ROOT / "artifacts/risk_model.joblib" | ||
|
|
||
|
|
||
| def create_app(model_path: Path = MODEL_PATH) -> Flask: | ||
| app = Flask(__name__) | ||
| CORS(app) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
src/api/app.pyto serve risk predictions based on transaction features.src/api/simple_api_server.pyto use risk probability instead of risk score.src/data/loader.pyfor SAML-D dataset.src/evaluation/metrics.pyandsrc/evaluation/explainability.py.src/features/behavioral_features.pyandsrc/features/transaction_features.py.src/models/train.pyandsrc/models/calibration.py.tests/test_features.pyandtests/test_inference.py.