Emit position_ids from the SFT collator when sequence parallelism is enabled

[qgallouedec]

Running SFT with the documented ALST/Ulysses setup (distributed_type: DEEPSPEED + parallelism_config_sp_backend: deepspeed, e.g. examples/accelerate_configs/alst_ulysses_4gpu.yaml) fails on the first batch:

ValueError: Ulysses SP requires `position_ids` in every dataloader batch so that each token
retains its correct global position after sequence sharding. ...

DataCollatorForLanguageModeling only emits position_ids in padding-free mode; the standard padded path never includes them, so the documented Ulysses configuration doesn't work out of
the box.

Fix

• DataCollatorForLanguageModeling gains return_position_ids (default False): in padded mode it then returns position_ids alongside attention_mask; per-sample arange, or boundary-resetting ids for packed examples (reusing the existing get_position_ids_from_packed_seq_lengths).
• SFTTrainer flips it on automatically when accelerator.parallelism_config.sp_enabled.

No behavior change for any existing configuration (flag defaults off; SP configs previously crashed).

Repro

ulysses2.yaml (2 GPUs):

compute_environment: LOCAL_MACHINE
deepspeed_config:
  zero_stage: 3
  seq_parallel_communication_data_type: bf16
  zero3_init_flag: true
distributed_type: DEEPSPEED
mixed_precision: bf16
num_machines: 1
num_processes: 2
parallelism_config:
  parallelism_config_dp_replicate_size: 1
  parallelism_config_dp_shard_size: 1
  parallelism_config_tp_size: 1
  parallelism_config_sp_size: 2
  parallelism_config_sp_backend: deepspeed
  parallelism_config_sp_seq_length_is_variable: true
  parallelism_config_sp_attn_implementation: sdpa

repro.py:

from datasets import Dataset
from trl import SFTConfig, SFTTrainer

dataset = Dataset.from_dict({"text": ["The quick brown fox jumps over the lazy dog. " * 20] * 16})
trainer = SFTTrainer(
    model="trl-internal-testing/tiny-Qwen3ForCausalLM",
    args=SFTConfig(max_steps=2, per_device_train_batch_size=1, pad_to_multiple_of=2, report_to=[]),
    train_dataset=dataset,
)
trainer.train()
print("training OK")

accelerate launch --config_file ulysses2.yaml repro.py

Before (main): the ValueError above at the first batch. After (this branch): training OK.

Tested

End-to-end on 8×H100 (sp_size=8, ZeRO-3, Qwen3-0.6B, 1,048,576-token sequences): before → the ValueError above; after → trains at 234 s/step, 24.3 GB/GPU (loss/metrics consistent with the ring-attention CP run of the same setup).

---

Note

Low Risk
Default-off flag and narrow auto-enable only when SP is active; fixes a crash path without changing standard SFT behavior.

Overview
Fixes Ulysses/ALST SFT setups that failed on the first batch because padded batches never included position_ids (only padding-free mode did).

DataCollatorForLanguageModeling adds optional return_position_ids (default False). In standard padded mode it can now return position_ids with attention_mask, using per-sequence arange or packed seq_lengths (same helper as padding-free). SFTTrainer sets return_position_ids=True when accelerator.parallelism_config.sp_enabled.

Tests cover padded and packed return_position_ids batches. Default configs are unchanged.

^{Reviewed by Cursor Bugbot for commit d13784a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d13784a7ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Guard access to the new Accelerate property

When TRL is installed with a supported Accelerate version below 1.10.1, constructing any SFTTrainer now raises AttributeError here because those releases predate Accelerator.parallelism_config; the project still declares accelerate>=1.4.0, and _compat.py explicitly handles the absence of parallelism support in these versions. Gate this access with an explicit Accelerate version check (or raise the minimum dependency) so ordinary non-parallel SFT remains usable.

AGENTS.md reference: AGENTS.md:L47-L48

Useful? React with 👍 / 👎.

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit d13784a. Configure here.}

[cursor]

SP check crashes on older Accelerate

High Severity

SFTTrainer now reads self.accelerator.parallelism_config and sp_enabled on every init. Those attributes do not exist on the supported Accelerate range: parallelism_config arrived in 1.10 and sp_enabled in 1.12, while TRL still requires and CI-tests accelerate>=1.4.0. That makes ordinary (non-SP) SFTTrainer construction raise AttributeError.

 

^{Reviewed by Cursor Bugbot for commit d13784a. Configure here.}