Import print_dist in auto_tp - #8311
Merged
delock merged 1 commit intoAug 25, 2026
Merged
Conversation
register_replicated_grad_hooks() calls print_dist() but auto_tp.py only imports log_dist, so the call raises NameError whenever any replicated parameter gets a grad all-reduce hook: NameError: name 'print_dist' is not defined DeepSpeedEngine calls this during AutoTP setup (engine.py:806, :845). Signed-off-by: alanhuangyoo <alanhuangyoo@gmail.com>
delock
approved these changes
Aug 25, 2026
Merged
via the queue into
deepspeedai:master
with commit Aug 25, 2026
b753aec
13 of 15 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AutoTP.register_replicated_grad_hooks()callsprint_dist(), butauto_tp.pyimports onlylog_dist:So the call raises as soon as
registeredis non-empty:DeepSpeedEnginecalls this during AutoTP setup (deepspeed/runtime/engine.py:806and:845), so any AutoTP run that registers a hook on a replicated parameter hits it.print_distandlog_distare both indeepspeed/utils/logging.pyand are not interchangeable —print_distexists specifically for messages that should appear regardless of log level — so this adds the import rather than switching the call tolog_dist.Before, on master:
After:
pre-commit run --files deepspeed/module_inject/auto_tp.pyis clean.The call came in with #8185 on 2026-08-20. It is not caught by the live CI:
modal-torch-latestrunstests/unit/v1/only, and the self-hosted GPU workflows that would rununit/have not produced a run in a long time (nv-a6000last ran 2025-08-01,nv-nightly2026-01-15,nv-torch-latest-v100andnv-inferencehave no runs listed).Env: torch 2.13.0+cu130, H20, single node.