Qwen3.6-35B-A3B: the penalties count the reply's tokens by default too; STRATA_PENALTY_PROMPT=1 brings the old window back (#4590)

The Qwen3.6 engine (src/qwen35/q36_engine.cu) started its penalty history from the whole prompt (`hist = ids`), so its model-card presence penalty 1.5 counted the prompt's last tokens as in the other engines. The history now starts at the last prompt token and grows with the tokens written (penalty_history_start in include/strata/qwen35/sampling.hpp), the same window as src/program/generate.cpp; batch slots copy that history. STRATA_PENALTY_PROMPT=1 keeps the whole prompt.

Tested: qwen35_sampling_test (276 checks, 9 new for the default and the opt-out), and a CUDA sm_86 build on z440. Replay on z440, one A4000, UD-Q6_K_XL, the reporter's conversation cut to 250 records, 400 tokens: 16 sampled seeds and 2 greedy requests per arm. This model did not return the old JSON in either arm (0/16 with the default, 0/16 with STRATA_PENALTY_PROMPT=1; 0/2 greedy in both), so this replay does not show a difference between the two windows; no answer in either arm started with the old JSON or a code fence.

docs/DETAILS.md: the sentence saying the Qwen3.6 path still counts the prompt is replaced.

docs/DETAILS.md

@@ -538,7 +538,7 @@ applies at the next start. Network, key, MCP and program keys are not editable t
   `penalty_last_n` (64 when a penalty is set and the request gives none) is a window over the tokens of the current answer: the
   prompt, including earlier assistant turns in it, is not counted, so the numbers or a quoted earlier answer in a new task are not
   penalised. `STRATA_PENALTY_PROMPT=1` counts the last `penalty_last_n` tokens of prompt and answer together, the old window.
-  This is the Qwen3.8 engine path; the Qwen3.6-35B-A3B path still counts the prompt.
+  This holds for the Qwen3.8 and Qwen3.6-35B-A3B engines on CUDA, HIP, Metal and SYCL.
 - Sampling defaults, in order of precedence: the request, the web page's shared Chat settings, the config's `sampling` block,
   then the model card. Without a block, thinking requests get temperature 1.0, top_p 0.95, top_k 20, and non-thinking
   requests get 0.7, 0.8, 20 with presence_penalty 1.5 (Qwen3.6-35B-A3B uses presence_penalty 1.5 in both). Greedy is only for

include/strata/qwen35/sampling.hpp

@@ -6,6 +6,8 @@
 #include <map>
 #include <vector>
 
+#include "strata/kernels/sampler.hpp"   // penalty_counts_prompt (declarations only; no GPU needed)
+
 namespace strata::qwen35 {
 
 struct Sampling {
@@ -33,4 +35,13 @@ inline void apply_penalties(std::vector<float>& lg, const Sampling& s, const std
     }
 }
 
+/// #4590: the history the penalties start from for one request.  By default only the current reply counts: the history starts
+/// at the last prompt token (the one the first decode step feeds in) and grows with the tokens written, so the prompt's own
+/// numbers and a quoted earlier answer are not penalised.  `counts_prompt` (STRATA_PENALTY_PROMPT=1) is the old window, the
+/// whole prompt.  Same window as src/program/generate.cpp.
+inline std::vector<int32_t> penalty_history_start(const std::vector<int32_t>& prompt, bool counts_prompt) {
+    if (counts_prompt || prompt.empty()) return prompt;
+    return std::vector<int32_t>(1, prompt.back());
+}
+
 }  // namespace strata::qwen35

src/qwen35/q36_engine.cu

@@ -3568,7 +3568,10 @@ struct Serve {
             return;
         }
         std::mt19937_64 rng(smp.seed);
-        std::vector<int32_t> hist = ids;
+        // #4590: the penalties count the reply's tokens (and the fed-back last prompt token), not the prompt; STRATA_PENALTY_PROMPT=1
+        // restores the old window.  `ids` is the request's full prompt whether or not the session was resumed.
+        static const bool pen_prompt = strata::kernels::penalty_counts_prompt(std::getenv("STRATA_PENALTY_PROMPT"));
+        std::vector<int32_t> hist = strata::qwen35::penalty_history_start(ids, pen_prompt);
         const auto d0 = Clock::now();
         long long produced = 0, offered = 0, accepted = 0;
         const char* finish = "length";

src/qwen35/sampling_test.cpp

@@ -107,6 +107,34 @@ int main() {
         check(row.empty(), "empty row");
     }
 
+    // #4590: the history a request's penalties start from.  Default: the last prompt token only (the reply's tokens join it);
+    // STRATA_PENALTY_PROMPT=1 (counts_prompt): the whole prompt, as before.
+    {
+        using strata::qwen35::penalty_history_start;
+        const std::vector<int32_t> prompt = {5, 12, 12, 7777, 7778, 50, 51};
+        check(!strata::kernels::penalty_counts_prompt(nullptr) && !strata::kernels::penalty_counts_prompt("0") &&
+                  strata::kernels::penalty_counts_prompt("1"), "the switch: unset and 0 reply only, 1 the old window");
+        const std::vector<int32_t> h = penalty_history_start(prompt, false);
+        check(h.size() == 1 && h[0] == 51, "default: the history starts at the last prompt token");
+        check(penalty_history_start(prompt, true) == prompt, "opt-out: the whole prompt");
+        check(penalty_history_start({}, false).empty(), "empty prompt");
+        // a penalty on the first pick: the task's token 12 is untouched by default, penalised with the old window
+        Sampling t;
+        t.present = 1.5f; t.last_n = 64;
+        std::vector<float> a(8000, 0.f), b(8000, 0.f);
+        apply_penalties(a, t, h);
+        apply_penalties(b, t, penalty_history_start(prompt, true));
+        check(a[12] == 0.f && a[7777] == 0.f && a[51] == -1.5f, "default: only the fed-back token is penalised");
+        check(b[12] == -1.5f && b[7777] == -1.5f && b[51] == -1.5f, "opt-out: the prompt's tokens are penalised");
+        // a reply that repeats itself is still penalised: tokens written join the history
+        std::vector<int32_t> hr = h;
+        hr.push_back(900); hr.push_back(900);
+        std::vector<float> c(8000, 0.f);
+        t.freq = 0.5f;
+        apply_penalties(c, t, hr);
+        check(c[900] == -(1.5f + 2 * 0.5f) && c[12] == 0.f, "default: the reply's own repeats are still penalised");
+    }
+
     std::printf("qwen35_sampling_test: %d checks passed\n", checks);
     return 0;
 }