Skip to content

Add TensorRT deployment and ResNet-50 detector support - #1

Open
haruto2002 wants to merge 2 commits into
playbox-dev:mainfrom
haruto2002:feat/r50
Open

Add TensorRT deployment and ResNet-50 detector support#1
haruto2002 wants to merge 2 commits into
playbox-dev:mainfrom
haruto2002:feat/r50

Conversation

@haruto2002

Copy link
Copy Markdown

Summary

Two additions from post-challenge work, one commit each:

  1. TensorRT deployment (trt/) — a reproducible toolchain that takes a
    Sparse4D checkpoint to a real-time FP16 engine.
  2. ResNet-50 detector support — a ready-to-use config for the public NGC
    sparse4d_warehouse_v2.2_r50 checkpoint, quality-weighted detection
    scoring, and documentation of the findings behind both.

Neither changes existing behavior: the paper's published numbers remain
exactly reproducible with the documented settings.

TensorRT deployment

  • Full chain .pth → ONNX → engine → inference → GT check, run entirely
    inside the vss-rt-cv container (host needs only docker + GPU driver).
  • ONNX export with automatic TAO spec repair, trtexec build wrapper
    (FP16 + MSDA plugin, dynamic camera count), engine inference with
    externally driven instance-bank recurrence, and a hybrid runner pairing
    the TRT detector with the PyTorch identity layer.
  • Includes a defense against an FP16 recurrent-cache NaN collapse
    (nan_to_num + clamp at the feedback boundary; reproducible with
    --no-clamp, also reported to NVIDIA).
  • 4.6× speedup at 16 cameras — 32.5 ms/step (30.8 FPS, real time) vs
    149.3 ms in PyTorch — for a 0.4–1.8 HOTA cost.
    Verified on all three
    validation scenes × 9,000 frames against the official 3D-HOTA protocol
    (trt/RESULTS.md).

ResNet-50 support & quality scoring

  • src/sparseqid/configs/sparse4d_rn50_v2.2: architecture config + anchors
    for the public rn50 checkpoint (use via --config-dir).
  • sqid infer --quality-score: scores detections as
    sigmoid(cls) × sigmoid(centerness) (the TAO decoder convention) instead
    of the class score alone. Opt-in; default behavior is unchanged.
  • docs/EXPERIMENTS.md records the post-challenge ablation study: the rn50
    detector is far more robust under domain shift (val GT-weighted HOTA
    25.64 → 35.01), float32 inference is worth several HOTA points over
    --amp bfloat16 (+4.6 on the hidden test set), and quality scoring adds
    ~+1. The full ladder reaches 38.69 on val and 55.9 on the hidden test set
    (submitted system: 38.01). docs/REPRODUCE.md documents the
    precision/reproducibility trade-off of --amp.

Testing

  • uv run pytest: 14 passed, 1 skipped (unchanged).
  • TRT chain verified end to end on Warehouse_020–022 (accuracy + timing in
    trt/RESULTS.md).
  • --quality-score output verified byte-identical to the externally
    validated driver used in the experiments.

haruto2002 added 2 commits September 1, 2026 19:18
Reproducible toolchain for deploying the Sparse4D detector with TensorRT,
run entirely inside the vss-rt-cv container: checkpoint-to-ONNX export with
automatic spec repair, a trtexec engine-build wrapper (FP16 + MSDA plugin,
variable camera count), engine inference with externally driven instance-bank
recurrence, a hybrid runner pairing the TRT detector with the PyTorch
identity layer, and a frame-aligned ground-truth check. Includes an FP16
recurrent-cache NaN-collapse defense at the feedback boundary, from-scratch
setup and results docs, and a 4.6x speedup at 16 cameras (30.8 FPS,
real time) for a 0.4-1.8 HOTA cost.
Ship an inference architecture config for the public NGC
sparse4d_warehouse_v2.2_r50 checkpoint (configs/sparse4d_rn50_v2.2), add
quality-weighted detection scoring to sqid infer (--quality-score:
sigmoid(cls) * sigmoid(centerness), the TAO decoder convention), document
the bfloat16 precision trade-off in REPRODUCE.md, and record the
post-challenge findings (detector robustness, precision, emission scoring,
ablation ladder, association headroom, runtime) in docs/EXPERIMENTS.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant