1 Commits
Author SHA1 Message Date
noon-at-cgn 48f2c47d54 batch-mtp: work with a layer split (slot drafters on the last stage's GPU, grouped slot rows through every stage)
- generate.cpp: --batch-mtp accepted with a layer split on separate GPUs (refused with --batch-groups); slot drafters
  load from the last stage's slot sessions under the last stage's device guard and bind to its weights/head; their
  residual buffers, row copy (async on the drafter's stream) and draft-KV copies run on the drafter's device; every
  stage verifier gets the 8-row window size and the batch graph limit; slot-drafter logits are not charged to CUDA0.
- verify.cpp: stage_batch's layout rules move to include/strata/core/batch_rows.hpp (CPU test batch_rows_test); grouped
  rows are allowed on a stage that hands rows on, the hand-off bound is kVerifyMaxT rows.
- docs/BATCHING.md, --batch-mtp help text.
2026-10-09 15:16:16 +02:00