tools/q8_dense_gguf.py: convert every shard that holds dense matrices

The tool converted only the shard it was given and hard-linked the others, which is enough for Flash-Next (every
natively served projection and the head are in shard 1) but not for Swift 1.5 IQ3_XXS, which keeps 78 of them in
shard 1 and 223 in shard 2: one run left three quarters of them in their K- / i-quants, silently, and the kernel
only runs Q8_0 matrices. Now any shard of the model may be named; every shard that holds such a matrix is written
as before (same tensor-info length, Q8_0 blocks appended, offsets checked) and the others are hard-linked.

Checked: Swift 1.5 IQ3_XXS in one run gives both shards byte-identical to two single-shard runs (301 tensors);
Flash-Next IQ3_XXS converts 301 tensors in shard 1 and links shard 2.

docs/OLDER_GPUS.md

@@ -108,9 +108,10 @@ and i-quant blocks (Q6_K, IQ4_XS, Q4_K, IQ3_S, ...) one int at a time. Two chang
   keep the usual kernel. Not bitwise the exact kernels (the per-32 products are added in another order). The packed
   copies cost their size in VRAM beside the GGUF layout the prompt path reads (Flash-Next: 2.9 GiB of dense
   projections over the stages and 0.6 GiB for the head).
-- The GSQ-RCO packs have no Q8_0 dense matrices, so the kernel needs a shard whose dense projections and head are
-  Q8_0: `python tools/q8_dense_gguf.py <model>-00001-of-00002.gguf <new folder>` writes one (+3.5 GiB; the experts keep
-  their offsets, so the native pack is reused; the other shards are hard-linked beside it). Point `--native` and
+- The GSQ-RCO packs have no Q8_0 dense matrices, so the kernel needs a copy of the model whose dense projections and head are
+  Q8_0: `python tools/q8_dense_gguf.py <model>-00001-of-00002.gguf <new folder>` writes one: every shard that holds
+  such matrices is converted (Swift 1.5 keeps them in both of its shards) and the rest are hard-linked beside them
+  (+3.5 GiB; the experts keep their offsets, so the native pack is reused). Point `--native` and
   `--ple-gguf` at the new folder. Requantizing costs at most 0.4% of a tensor's |w|max (Q8_0 is finer than the
   sources).
 - IQ3_XXS's codebook and sign-mask tables are staged in shared memory by the grouped expert kernels on compute

tools/q8_dense_gguf.py

@@ -1,8 +1,10 @@
-"""tools/q8_dense_gguf.py - a copy of a model's first shard whose natively served dense projections and head are Q8_0,
-for the Pascal GP100 decode kernel (STRATA_Q8_SM60=1, docs/OLDER_GPUS.md).
+"""tools/q8_dense_gguf.py - a copy of a model whose natively served dense projections and head are Q8_0, for the
+Pascal GP100 decode kernel (STRATA_Q8_SM60=1, docs/OLDER_GPUS.md).
 
 The GSQ-RCO packs keep those matrices in K- and i-quants (Q6_K, IQ4_XS, Q4_K, IQ3_S, Q5_K, Q2_0, IQ4_NL), whose
-decode is what a GP100 die is slowest at; STRATA_Q8_SM60 only runs Q8_0 matrices. This writes:
+decode is what a GP100 die is slowest at; STRATA_Q8_SM60 only runs Q8_0 matrices. Every shard of the model is
+handled, because a model may keep dense matrices in more than one (Swift 1.5 IQ3_XXS: 78 in shard 1, 223 in
+shard 2). A shard that holds any of them is written as:
 
   * the tensor-info block with the SAME byte length: names and dims are unchanged, only the u32 type and the u64
     offset of each converted tensor change, so the data section starts where it did and every other tensor - the
@@ -12,12 +14,13 @@ decode is what a GP100 die is slowest at; STRATA_Q8_SM60 only runs Q8_0 matrices
     Qwen3.8-Flash-Next IQ3_XXS: 301 tensors, +3.5 GiB, the worst requantization error 0.41% of a tensor's |w|max
     (Q8_0 is finer than every source format).
 
-The other shards of the model are hard-linked into the output folder (--copy copies them): the engine finds them
-beside --native, so --ple-gguf must name the one in the new folder too (it refuses two shard 2s).
+A shard that holds none is hard-linked into the output folder (--copy copies it): the engine finds the shards beside
+--native, so --native and --ple-gguf must both name files in the new folder (it refuses two shard 2s).
 
     python tools/q8_dense_gguf.py <folder>/<model>-00001-of-00002.gguf <new folder> [--copy]
 
-Then point the config's --native and --ple-gguf at the new folder and run the engine with STRATA_Q8_SM60=1.
+Any shard of the model may be named. Then point the config's --native and --ple-gguf at the new folder and run the
+engine with STRATA_Q8_SM60=1.
 """
 from __future__ import annotations
 
@@ -59,23 +62,17 @@ def to_f32(t) -> np.ndarray:
     return np.asarray(dequantize(raw, t.tensor_type), dtype=np.float32).reshape(ne1, ne0)
 
 
-def main() -> int:
-    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
-    ap.add_argument("shard1", type=Path, help="the model's first GGUF shard (the one --native names)")
-    ap.add_argument("out_dir", type=Path, help="folder for the new shard 1 and the other shards' links")
-    ap.add_argument("--copy", action="store_true", help="copy the other shards instead of hard-linking them")
-    a = ap.parse_args()
-    src = a.shard1.resolve()
-    out = a.out_dir.resolve()
-    if out == src.parent:
-        sys.exit("the output folder must not be the model's own folder")
-    dst = out / src.name
+def shards_of(path: Path) -> list[Path]:
+    """Every shard of the model `path` belongs to (just `path` for an unsplit file)."""
+    m = re.match(r"(.*-)(\d{5})-of-(\d{5})(\.gguf)$", path.name)
+    if not m:
+        return [path]
+    return [path.parent / f"{m[1]}{k:05d}-of-{m[3]}{m[4]}" for k in range(1, int(m[3]) + 1)]
 
-    t0 = time.time()
-    r = GGUFReader(src)
-    align, data_start, src_size = r.alignment, r.data_offset, os.path.getsize(src)
-    mm = r.data
-    jobs = []
+
+def jobs_of(r) -> list:
+    """The shard's convertible tensors and where each one's type/offset fields sit in its tensor-info block."""
+    data_start, mm, jobs = r.data_offset, r.data, []
     for t in r.tensors:
         if not is_target(t.name) or int(t.tensor_type) == int(QT.Q8_0):
             continue
@@ -89,8 +86,12 @@ def main() -> int:
             print(f"  {t.name}: rows of {int(t.shape[0])} are not whole Q8_0 blocks, kept")
             continue
         jobs.append((t, p))
-    print(f"{src.name}: {len(r.tensors)} tensors, {len(jobs)} to convert to Q8_0", flush=True)
+    return jobs
 
+
+def convert(src: Path, dst: Path, r, jobs) -> int:
+    t0 = time.time()
+    align, data_start, src_size = r.alignment, r.data_offset, os.path.getsize(src)
     blobs, worst = [], 0.0
     for t, p in jobs:
         f = to_f32(t)
@@ -99,10 +100,9 @@ def main() -> int:
         back = dequantize(q, QT.Q8_0).reshape(f.shape)
         worst = max(worst, float(np.abs(back - f).max() / max(float(np.abs(f).max()), 1e-30)))
         blobs.append((t, p, q))
-    print(f"converted in {time.time() - t0:.0f} s; worst requantization error {worst:.2e} of a tensor's |w|max; "
+    print(f"  converted in {time.time() - t0:.0f} s; worst requantization error {worst:.2e} of a tensor's |w|max; "
           f"{sum(b[2].nbytes for b in blobs) / 2**30:.2f} GiB to append", flush=True)
 
-    out.mkdir(parents=True, exist_ok=True)
     tmp = dst.with_name(dst.name + ".part")
     shutil.copyfile(src, tmp)
     with open(tmp, "r+b") as fh:
@@ -125,20 +125,42 @@ def main() -> int:
             assert t.tensor_type == QT.Q8_0 and np.array_equal(np.asarray(t.data).view(np.uint8).reshape(-1), conv[t.name]), t.name
         else:
             assert t.tensor_type == orig[t.name].tensor_type and t.data_offset == orig[t.name].data_offset, t.name
-    print(f"wrote {dst} ({os.path.getsize(dst) / 2**30:.2f} GiB): {len(conv)} converted, {len(v.tensors) - len(conv)} "
+    print(f"  wrote {dst} ({os.path.getsize(dst) / 2**30:.2f} GiB): {len(conv)} converted, {len(v.tensors) - len(conv)} "
           "untouched, offsets checked", flush=True)
+    return len(conv)
 
-    m = re.match(r"(.*-)(\d{5})-of-(\d{5})(\.gguf)$", src.name)
-    if m:
-        for k in range(1, int(m[3]) + 1):
-            name = f"{m[1]}{k:05d}-of-{m[3]}{m[4]}"
-            if k == int(m[2]) or (out / name).exists():
-                continue
-            if a.copy:
-                shutil.copyfile(src.parent / name, out / name)
-            else:
-                os.link(src.parent / name, out / name)
-            print(f"{'copied' if a.copy else 'linked'} {name}")
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("shard", type=Path, help="any GGUF shard of the model (or the model's only file)")
+    ap.add_argument("out_dir", type=Path, help="folder for the converted shards and the others' links")
+    ap.add_argument("--copy", action="store_true", help="copy the shards with nothing to convert instead of linking them")
+    a = ap.parse_args()
+    out = a.out_dir.resolve()
+    shards = shards_of(a.shard.resolve())
+    if out == shards[0].parent:
+        sys.exit("the output folder must not be the model's own folder")
+    missing = [s for s in shards if not s.is_file()]
+    if missing:
+        sys.exit(f"missing shard(s): {', '.join(str(s) for s in missing)}")
+    out.mkdir(parents=True, exist_ok=True)
+    total = 0
+    for src in shards:
+        dst = out / src.name
+        r = GGUFReader(src)
+        jobs = jobs_of(r)
+        print(f"{src.name}: {len(r.tensors)} tensors, {len(jobs)} to convert to Q8_0", flush=True)
+        if jobs:
+            total += convert(src, dst, r, jobs)
+            continue
+        if dst.exists():
+            dst.unlink()                                     # a link or copy from an earlier run
+        if a.copy:
+            shutil.copyfile(src, dst)
+        else:
+            os.link(src, dst)
+        print(f"  {'copied' if a.copy else 'linked'} (nothing to convert)", flush=True)
+    print(f"{total} tensors converted over {len(shards)} shard(s) in {out}")
     return 0