A should-cost estimation system for discrete manufactured parts (CNC machining, injection molding, sheet metal, die casting). It combines a transparent parametric cost engine with a machine-learned layer that predicts what a real supplier quote is likely to look like, flags quotes that look like outliers, and explains why.
Should-costing is the practice of independently estimating what a part should cost to make — from material, process, and labor logic — so a buyer has a baseline to negotiate against instead of just trusting whatever number a supplier sends back. It's a real, recurring problem in sourcing and DFM (design-for-manufacturability) work; this project is a from-scratch implementation of that workflow, not a wrapper around someone else's tool.
On the data: every number in this repo comes from a synthetic dataset generated by a documented parametric model (see Data). No client, employer, or proprietary quote data is used anywhere.
Two layers, kept deliberately separate:
- Analytical engine (
cost_engine/engine/) — deterministic, fully interpretable. Given a part spec and the assumptions incost_engine/config/cost_assumptions.yaml, it returns a full cost breakdown (material / machine / labor / tooling / overhead / margin). No training, no data dependency — it's just engineering-economics logic and it's always available, even for a part type the ML model has never seen priced. - Learned layer (
cost_engine/ml/,cost_engine/analysis/) — trained on a large synthetic dataset of (part spec → engine should-cost → simulated supplier quote) rows. Its job isn't to reproduce the engine — it's to predict the systematic bias and variance real quotes show around the should-cost baseline, give a calibrated uncertainty range, and flag quotes that don't fit the pattern.
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 1. generate the synthetic dataset (~8,000 rows, a few seconds)
python -m cost_engine.data.generator --n-rows 8000 --seed 7 --out data/generated/quotes_dataset.csv
# 2. train + evaluate all three models, run outlier detection, build every
# plot in this README from scratch
python scripts/run_demo.py
# 3. estimate a single part
python -m cost_engine.cli estimate --process cnc_machining --material aluminum_6061 \
--mass-g 220 --lot-size 500 --tolerance-class precision \
--surface-finish as_machined --removed-volume-cm3 140 --explain
# 4. estimate a whole BOM/RFQ file
python -m cost_engine.cli batch --bom data/sample_bom.csv --out /tmp/bom_estimates.csvRun the test suite with python -m pytest tests/ -q (15 tests, no network
or pretrained model needed — they train small models on the fly).
$ python -m cost_engine.cli estimate --process cnc_machining --material aluminum_6061 \
--mass-g 220 --lot-size 500 --tolerance-class precision \
--surface-finish as_machined --removed-volume-cm3 140 --explain{
"should_cost_breakdown": {
"material_cost": 0.93, "machine_cost": 9.41, "labor_cost": 1.88,
"tooling_cost_amortized": 0.0, "overhead_cost": 3.42,
"margin_sga_cost": 2.81, "total_should_cost": 18.45
},
"assumption_uncertainty": {"p10": 15.32, "p50": 18.47, "p90": 21.69},
"ml_quote_prediction": {
"predicted_quote_p10": 16.25, "predicted_quote_p50": 19.65, "predicted_quote_p90": 23.37
},
"explanation_top_factors": [
{"feature": "log_total_should_cost", "shap_contribution": -0.27},
{"feature": "removed_volume_cm3", "shap_contribution": 0.017},
{"feature": "log_lot_size", "shap_contribution": 0.013}
]
}Two different uncertainty ranges are reported on purpose:
assumption_uncertainty— Monte Carlo over the engine's own assumptions (material price, machine rate, cycle time, overhead %, labor rate each perturbed randomly, 2,000 draws). Answers: how sensitive is my should-cost estimate to the assumptions I plugged in?ml_quote_prediction— P10/P50/P90 from gradient-boosted quantile models trained on the simulated quote dataset. Answers: given how real quotes have varied historically, what should I expect a supplier to actually say?
cost_engine/engine/sensitivity.py also produces a full tornado breakdown per
factor — see assets/sensitivity_tornado.png for the same example part.
Three models predict quoted_cost, evaluated on the same 20% held-out
split (1,600 rows), stratified by process:
| Model | MAE | MAPE | R² |
|---|---|---|---|
engine_baseline — should-cost × one calibration factor per process |
$27.07 | 16.28% | 0.964 |
ridge — log-linear model on the full feature set |
$26.69 | 15.95% | 0.965 |
hist_gbm — gradient-boosted trees on the full feature set |
$28.87 | 16.62% | 0.951 |
The nonlinear model doesn't win here, and that's the honest result, not
a bug. The synthetic quote generator (cost_engine/data/generator.py) applies a
per-process multiplicative bias plus lognormal noise to the engine's
should-cost — which means the true relationship is, by construction, close
to log-linear in should_cost and process. Ridge is a near-perfect
functional match for that; HistGBM's extra flexibility has nothing real to
fit and just adds variance. The SHAP summary below confirms it directly —
log_total_should_cost swamps every other feature:
So why keep hist_gbm in the repo at all? Two reasons: (1) it's what makes
the P10/P90 quantile interval possible — Ridge doesn't natively produce a
calibrated, heteroscedastic uncertainty band, and quote spread in this
dataset genuinely does vary by tolerance class; (2) running this comparison
is the deliverable — knowing when the fancier model isn't worth it is as
useful a skill as building it. Per-process metrics (where the picture is
similar) are in assets/results.json.
Target: an 80% (P10–P90) interval should contain the true quote 80% of the time. Measured on the held-out set: 72.3% — the model is somewhat overconfident (intervals too narrow). Documented in Limitations rather than hidden; conformal calibration is the natural next step.
cost_engine/analysis/outlier_detection.py flags quotes whose deviation from the
model's expected price (per-process z-score on the log ratio) exceeds a
threshold. Because the generator injects known outlier quotes
(is_synthetic_outlier, ~5% of rows, never seen by the model), the detector
can be scored against ground truth instead of just eyeballed:
| Metric | Value |
|---|---|
| Precision | 72.7% |
| Recall | 38.1% |
| F1 | 0.50 |
| True outliers in test set | 84 / 1,600 |
Recall is the weaker number — at z >= 2.5 the detector is conservative by
design (fewer false alarms for a human reviewer to chase). Lowering the
threshold trades precision for recall; see Limitations.
cost_engine/config/cost_assumptions.yaml material $/kg, machine rates, labor, overhead — all editable, nothing hardcoded
cost_engine/engine/ the deterministic should-cost engine + sensitivity analysis
cost_engine/data/ BOM/RFQ schema + parsing, synthetic dataset generator
cost_engine/ml/ feature engineering, training, evaluation, SHAP explanations
cost_engine/analysis/ outlier detection, scored against ground truth
cost_engine/cli.py estimate / batch / sensitivity commands
scripts/run_demo.py regenerates every number and plot in this README from scratch
scripts/build_notebook.py regenerates notebooks/01_exploratory_analysis.ipynb
data/ dataset docs + a 10-line sample BOM for the batch demo
tests/ 15 tests covering the engine, generator, schema, and ML pipeline
assets/ architecture diagram, plots, results.json
The dataset is fully synthetic; see data/README.md for
exactly how it's generated and why. In short: sampled part specs run through
the deterministic engine, then a documented noise model simulates realistic
supplier quote behavior around that baseline. No proprietary or client cost
data — mine or anyone else's — appears anywhere in this repository.
- Interval calibration is off (72.3% vs. target 80%) — worth fixing with conformal prediction before trusting the band for anything real.
- The synthetic noise model is intentionally simple (per-process bias + lognormal spread). Real supplier quotes have more structure — regional cost differences, capacity cycles, relationship pricing — that a richer simulator (or, better, real anonymized quote history) would surface.
- Outlier recall is conservative. A precision/recall curve across z-thresholds (rather than one fixed cutoff) would let a user pick their own tolerance for false alarms vs. missed outliers.
- Cost assumptions are illustrative, not live market data — the whole point of keeping them in one YAML file is that they're meant to be replaced with real numbers for real use.
- No CAD ingestion. Geometry features (
removed_volume_cm3, cavity/bend counts) are inputs today; estimating them from an actual STEP/STL file would remove the main piece of manual BOM entry.
MIT — see LICENSE.