modelscope/FunASR
1
0
Fork 0
mirror of https://github.com/modelscope/FunASR.git synced 2026-10-12 02:19:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(train): align DDP no_sync with gradient accumulation (#3472)

Browse Source 
Synchronize DDP/FSDP gradients on the final microbatch of each accumulation window.

Verified with a two-rank CPU/Gloo reproduction, py_compile, diff checks, and a clean synthetic merge against current main.
This commit is contained in:
May
2026-08-05 11:32:24 +08:00
committed by GitHub
parent 6829ba7845
commit d410a5652d
1 changed files with 3 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
funasr/train_utils/trainer_ds.py
+3 -1
View File
@@ -648,7 +648,9 @@ class Trainer:
my_context = nullcontext
if self.use_ddp or self.use_fsdp:
my_context = model.no_sync if batch_idx % accum_grad != 0 else my_context
my_context = (
model.no_sync if (batch_idx + 1) % accum_grad != 0 else my_context
)
with my_context():
time2 = time.perf_counter()