PulseWatch — Time Series Anomaly Detection for Network Telemetry

Forecasting and anomaly detection on messy, real-world-style network traffic data — built to mimic what you'd actually pull off a fleet of edge routers / a Prometheus exporter, including scrape failures, duplicate scrapes, unit bugs, and sensor noise. Detects both a DDoS-style traffic spike and a subtle, periodic C2-style "beaconing" pattern hidden in the noise.

Why this exists

Production telemetry is never clean. This project treats that as the point: the pipeline doesn't just forecast a tidy signal, it first has to survive missing samples, out-of-order writes, duplicate rows, and a unit-conversion bug — the exact failure modes you hit with real monitoring stacks.

Pipeline

generate_data.py   -> simulates 30 days of 1-min traffic telemetry, injects
                       messiness + two labeled ground-truth anomalies
clean.py            -> sorts, dedups, fixes unit drift, resamples to a
                        strict grid, interpolates short gaps, robustly
                        clips impossible values (MAD-based, not std-based,
                        so real anomalies survive)
forecast.py          -> Holt-Winters (triple exponential smoothing) with
                        daily seasonality; evaluated on a 2-day hold-out
anomaly.py            -> two detectors: forecast-residual z-score (catches
                        sharp spikes) + IsolationForest on rolling-window
                        /periodicity features (catches subtle periodic
                        beaconing that residuals miss)
visualize.py          -> raw-vs-clean, forecast-vs-actual, and full-series
                        anomaly overview plots

Results

• Forecast accuracy (2-day hold-out): MAE 9.45 req/s, RMSE 11.73 req/s, MAPE 13.0%
• Cleaning pipeline: recovered a strict 1-min grid from raw data containing 430 duplicate timestamps, a 90-minute unit-conversion bug, ~1,500 NaNs, and 15 sensor glitches — fully logged and auditable (outputs/cleaning_log.txt)
• Detector comparison is the interesting part: the residual-based detector nails the DDoS spike but completely misses the beaconing pattern (it barely moves the raw value). The IsolationForest detector, using rolling-window and periodicity features, catches both — a concrete illustration of why a single detection method isn't enough for security-relevant time series.

Stack

Python, pandas, NumPy, statsmodels (Holt-Winters), scikit-learn (IsolationForest), matplotlib.

Run it

pip install pandas numpy statsmodels scikit-learn matplotlib
python data/generate_data.py
python src/clean.py
python src/forecast.py
python src/anomaly.py
python src/visualize.py

---

CV bullet points (pick 1-2)

PulseWatch — Time Series Anomaly Detection for Network Telemetry Built an end-to-end pipeline to forecast network traffic and detect security-relevant anomalies (DDoS spikes, periodic C2-style beaconing) in intentionally messy telemetry data (missing samples, duplicate scrapes, unit-conversion bugs). Implemented a robust MAD-based cleaning pipeline, a Holt-Winters seasonal forecaster (13% MAPE), and a dual anomaly detector combining residual analysis with IsolationForest on periodicity features — demonstrating why a single detection method misses subtle attack patterns that a feature-engineered ML detector catches.

Shorter version:

Designed and built a time series forecasting + anomaly detection pipeline (Python, statsmodels, scikit-learn) on messy network telemetry data, combining statistical and ML-based detectors to catch both volumetric (DDoS) and low-signal periodic (C2 beaconing) anomalies.