mirror of
https://github.com/gufo-org/gufo.git
synced 2026-11-07 09:35:59 -10:00
fix(ds4): qualify DSpark and refresh reference benchmarks (#250)
* docs(ds4): prepare benchmarks with optional pinned references * fix(ds4): qualify DSpark and refresh reference benchmarks
This commit is contained in:
Francesco Bozzo authored and GitHub committed
1 parent
4180fb3ddb
commit
2b9433d7f5
48 files changed
+13103
-1181
No files matched your search
@@ -31,11 +31,14 @@ runtime libraries. Skip loading if neither method is available.
|
||||
|
||||
## Measurement model
|
||||
|
||||
Everything is measured **over HTTP on both sides** so the prompt, timed scope
|
||||
Use **HTTP on both sides** by default so the prompt, timed scope
|
||||
and transport are identical: Gufo through `gufo serve`, the reference through
|
||||
its own OpenAI-compatible server. `gufo bench` and the standalone CLIs are
|
||||
kernel-iteration tools; their numbers are not published in the comparison
|
||||
tables.
|
||||
kernel-iteration tools unless the user authorizes a native reference benchmark.
|
||||
For DeepSeek, the user permits `ds4-bench` single-session AR/DSpark comparisons.
|
||||
Match prompt tokens, prefill interval, decode accounting and cache frontier first;
|
||||
label the native timing scope. Never silently compare it with whole-request HTTP
|
||||
wall time. `ds4-bench` does not implement C>1.
|
||||
|
||||
The driver is `tools/bench/model-bench.py`. `docs/models/<model>/artifacts/bench.json`
|
||||
declares the category, variant ids, sweep grid, workloads and reference server
|
||||
@@ -50,7 +53,7 @@ nix build # production Gufo binary at result/bin/gufo
|
||||
FILES="--gguf q4=/path/to/target-q4.gguf --draft q4=/path/to/draft-q4.gguf"
|
||||
nix develop -c python3 tools/bench/model-bench.py --model <model> tables # ids and existing artifacts
|
||||
nix develop -c python3 tools/bench/model-bench.py --model <model> $FILES run --target gufo --table single-ar-q4,single-dflash2-q4
|
||||
nix develop -c python3 tools/bench/model-bench.py --model <model> $FILES run --target reference --table single-ar-q4,single-dflash2-q4
|
||||
nix develop -c nix shell .#llama-cpp-reference -c python3 tools/bench/model-bench.py --model <model> $FILES run --target reference --table single-ar-q4,single-dflash2-q4
|
||||
nix develop -c python3 tools/bench/model-bench.py --model <model> render # or render --check
|
||||
```
|
||||
|
||||
@@ -64,7 +67,7 @@ refresh so an interrupted run cannot leave mixed-date rows; `--repetitions N`
|
||||
overrides every table's repetition count (mean ± sd above 1);
|
||||
When a reference server is killed mid-table (SIGKILL is the kernel OOM
|
||||
killer on this host), the driver marks that row and every deeper depth or
|
||||
larger concurrency `unavailable` in the artifact and `render` shows `n/a`
|
||||
larger concurrency `unavailable` in the artifact and `render` shows `N/A`
|
||||
for them, since they are a host limit rather than missing work.
|
||||
`--depths 0,4096` restricts single-user tables to those depths and
|
||||
`--mode ar` / `--mode <spec>` restricts to one mode (use it to skip a
|
||||
@@ -102,8 +105,16 @@ against); `tables` keyed by table id with that table's workload, and
|
||||
`gufo.serve` / `reference.server` + `args` for the two servers. Readiness is
|
||||
`GET /ready` on Gufo and `GET /health` on llama-server.
|
||||
|
||||
Reference servers (llama.cpp, audio.cpp, stable-diffusion.cpp) come from this
|
||||
repository's `flake.nix`; do not use ad-hoc installs.
|
||||
Select reference runtimes explicitly: `nix shell .#llama-cpp-reference` for
|
||||
Qwen AR/DFlash2, `.#llama-cpp-mtp-reference` for Flash-Next MTP, or
|
||||
`.#ds4-reference` for DeepSeek AR/DSpark. Combine the two llama packages when
|
||||
refreshing both modes. None is part of Gufo, the default dev shell or hosted
|
||||
checks. Preserve the pinned recipes under `.devops/nix`; do not use ad-hoc builds.
|
||||
Antirez readiness is `/v1/models`. Its HTTP payload lacks stage timings;
|
||||
`tools/ds4/server_metrics.py` reads the pinned server's existing prompt-done
|
||||
and final-decode timers, verifies token counts and retains the per-request
|
||||
cohort durations. Do not substitute client wall time or infer client IDs from
|
||||
interleaved log lines. Qualify prepared-prefix reuse before publishing C>1.
|
||||
|
||||
Request-level timing comes from the response, not the client clock, whenever
|
||||
the server provides it: Gufo and llama-server both return llama.cpp-compatible
|
||||
@@ -124,10 +135,10 @@ and the table says so.
|
||||
enter Git.
|
||||
2. Same input, same artifact, same timed scope on both sides: same GGUF or
|
||||
safetensors, same prompt or audio file, same output length, same
|
||||
seed/temperature, same context depth, same concurrency, fresh server per
|
||||
point, no prompt-cache hits unless the cell says so. If the reference cannot
|
||||
match (different quantization, no equivalent speculative mode, no batching),
|
||||
measure the closest configuration and state the mismatch under the table.
|
||||
seed/temperature, same context depth and same concurrency. Start a fresh
|
||||
server per depth sweep or concurrency point; allow only the prefix reuse
|
||||
declared by the workload. If a missing reference feature prevents a close
|
||||
match, mark those reference cells `N/A` and explain why.
|
||||
3. Warm once, then time. Prepared decode cohorts use one output token to
|
||||
finish all prompts before timing tg128. Other corpus warmups use only the
|
||||
first concurrent group, capped at 16 output tokens. Single-user
|
||||
@@ -140,16 +151,20 @@ and the table says so.
|
||||
`context` to raise. Run model jobs sequentially; nothing else on the GPU.
|
||||
4. Before publishing a speed refresh of unchanged code, run the model's fast
|
||||
correctness suite when it has one (Qwen3.8 27B: `nix develop -c python3
|
||||
tools/qwen27b/check.py fast`; Qwen3.8-Flash-Next and DeepSeek V4 Flash:
|
||||
none today — say so in the card and rely on the concurrency `Exact`
|
||||
checks); the full `EVALUATION.md` suites are for changed kernels or
|
||||
models. Greedy speculative output must match AR token
|
||||
tools/qwen27b/check.py fast`; DeepSeek: `nix develop -c tools/ds4/check.py fast`;
|
||||
Flash-Next has no dedicated fast model suite). Contract tests and concurrency
|
||||
hashes do not replace independent model qualification; the full
|
||||
`EVALUATION.md` suites are for changed kernels or models. Greedy speculative output must match AR token
|
||||
IDs in the model checks. HTTP runs retain completion hashes: concurrency
|
||||
compares them against the Gufo AR C1 reference, while matching single-user
|
||||
AR/speculative depth rows allow the same text-consistency check. These hashes
|
||||
do not prove original-model accuracy. A mismatch or device fault is not a result.
|
||||
5. A cell without a current qualified measurement is `TODO`, never a stale
|
||||
number. Keep dates per table; do not sum stage medians into a headline.
|
||||
Qualify the reference's speculative path against its own AR on the matched
|
||||
pp2048 control before a full sweep, including prepared C1 reuse. If that
|
||||
equivalence fails, retain compact evidence and mark the comparison `N/A`.
|
||||
5. A supported cell without a current qualified measurement is `TODO`, never a
|
||||
stale number; an unsupported comparison is `N/A`. Keep measurement dates;
|
||||
do not sum stage medians into a headline.
|
||||
6. Retained JSON goes to `docs/models/<model>/artifacts/`; raw samples,
|
||||
traces, audio, images and videos stay in the ignored top-level `artifacts/`.
|
||||
|
||||
@@ -193,15 +208,14 @@ Every headline table carries the reference project next to Gufo:
|
||||
- `TODO` on either side leaves `Gain` as `TODO`.
|
||||
- A reference cell the host cannot produce for hardware reasons — the
|
||||
reference is OOM-killed at the required context, the model does not fit —
|
||||
is `n/a`, not `TODO`. A missing software feature (the reference cannot
|
||||
load a sidecar yet) stays `TODO` with the upstream reference noted, since
|
||||
a newer pin can fill it. Declare `n/a` cells in `docs/models/<model>/artifacts/unavailable.json`
|
||||
is `N/A`. Missing reference features that prevent a close comparison are also
|
||||
`N/A`, with a concise reason. `TODO` means a supported measurement is pending. Declare `N/A` cells in `docs/models/<model>/artifacts/unavailable.json`
|
||||
(`{"<table id>": {"<row label>" | "*": "<reason>"}}`; the suffix
|
||||
`-speculative` on a concurrency table id targets only its speculative
|
||||
reference column) and state the reason under the table. `render` prints
|
||||
`n/a` there and in the matching Gain cell.
|
||||
`N/A` there and in the matching Gain cell.
|
||||
- Keep AR comparisons in the dedicated AR table. If the reference lacks a
|
||||
speculative mode, leave that mode's cells TODO; do not launch another AR
|
||||
speculative mode, mark that mode's cells N/A; do not launch another AR
|
||||
sweep for each speculative workload. Existing combined-mode cards label
|
||||
their fallback explicitly as `Gain vs <ref> AR`.
|
||||
- One table per quantization and per mode; never pack `pp / tg` pairs or two
|
||||
@@ -217,13 +231,13 @@ around them (dates, acceptance notes, caveats) is hand-maintained.
|
||||
## LLM (GGUF text and vision models)
|
||||
|
||||
Models: `deepseek-v4-flash`, `qwen3.8-27b`, `qwen3.8-flash-next`.
|
||||
Reference: **llama.cpp** `llama-server`, same GGUF, ROCm build,
|
||||
Qwen reference: **llama.cpp** `llama-server`, same GGUF, ROCm build,
|
||||
`-ngl 999 -fa on --cache-reuse 0 --cache-ram 0 --jinja --reasoning off`,
|
||||
`-np C` and total `-c` covering every session's context. `--cache-ram 0`
|
||||
disables llama-server's separate host snapshot cache; live slot prefix reuse
|
||||
remains available. On a unified-memory host, avoiding that extra allocation
|
||||
can decide whether a deep sweep fits. When a reference row
|
||||
still dies, check `free` and the kernel log before recording `n/a` — a run
|
||||
still dies, check `free` and the kernel log before recording `N/A` — a run
|
||||
that finishes only by evicting and re-reading its mapped weights is not a
|
||||
speed measurement either.
|
||||
Gufo: `gufo serve llm --think off --max-pending-per-client 8`, greedy, seed
|
||||
@@ -232,6 +246,17 @@ Gufo: `gufo serve llm --think off --max-pending-per-client 8`, greedy, seed
|
||||
only for the image-encoder table. The sweep parameters are identical across
|
||||
the three models so the documents stay comparable.
|
||||
|
||||
DeepSeek reference: **antirez/ds4**, pinned by `.#ds4-reference`, same target
|
||||
and DSpark GGUF. The server uses `--rocm --ctx <per-session-capacity>`;
|
||||
AR C>1 adds `--batched-session C`. At pin `0aaea5a2`, any `--batched-session`
|
||||
value disables DSpark on ROCm, including 1. Omit it for C1 DSpark; keep C>1
|
||||
DSpark reference cells N/A until a build supports them. DSpark adds
|
||||
`--dspark --mtp-model <support>`. Disable
|
||||
thinking in each HTTP request. Native `ds4-bench` supports the sidecar but
|
||||
uses one session; its `ctx_tokens` is the post-prefill frontier, not cached
|
||||
depth, and `prefill_tokens` is the increment from the preceding frontier.
|
||||
Reject short/EOS-truncated rows and retain full `gen_tps`, not steady-only rates.
|
||||
|
||||
Tables (append `-<quant>` for several quantizations, e.g. `single-ar-q4`);
|
||||
place loading immediately before memory in the rendered card:
|
||||
|
||||
@@ -268,7 +293,7 @@ place loading immediately before memory in the rendered card:
|
||||
pp. The renderer selects the highest pp per engine/depth, preserving that
|
||||
measurement's standard deviation, and recalculates pp gain from the maxima.
|
||||
llama.cpp runs the same draft file through `--spec-type draft-dflash`,
|
||||
`draft-mtp` or `draft-dspark` (`speculative.reference.args` in
|
||||
`draft-mtp` (`speculative.reference.args` in
|
||||
`bench.json`, otherwise llama.cpp's defaults; tune them only when the
|
||||
reference project documents better values, and record the change). When
|
||||
the pinned llama.cpp cannot load the sidecar (Qwen3.8-Flash-Next MTP with
|
||||
@@ -277,11 +302,11 @@ place loading immediately before memory in the rendered card:
|
||||
for the speculative cells only — `flake.nix` exposes `llama-server-mtp`
|
||||
from the pinned ggml-org/llama.cpp#28243 branch — and the card must say the
|
||||
speculative reference comes from that branch. Without such a build,
|
||||
record the error once and leave its speculative cells `TODO`.
|
||||
record the error once and mark its speculative cells `N/A`.
|
||||
`Model / depth | Gufo pp (tok/s) | llama.cpp pp (tok/s) | Gain pp | Gufo tg mixed (tok/s) | llama.cpp tg mixed (tok/s) | Gain mixed | Gufo tg repetitive (tok/s) | llama.cpp tg repetitive (tok/s) | Gain repetitive`.
|
||||
Retain proposed/accepted draft counts in artifacts, never in card columns
|
||||
or figures. Do not replace a missing speculative reference with another AR
|
||||
sweep; leave its cells TODO and explain the missing support.
|
||||
sweep; mark its cells `N/A` and explain the missing support.
|
||||
3. **Multiple users, autoregressive**, followed by **4. Multiple users,
|
||||
speculative**. Use the **same pp2048 prompts as single-user d0**,
|
||||
tg128 and `C = 1,2,4,6,8`; context capacity 4096 per user is sufficient.
|
||||
@@ -305,6 +330,10 @@ place loading immediately before memory in the rendered card:
|
||||
distinct `id_slot` values in both phases. Validate that each measured
|
||||
request reuses the entire prompt (at most four final tokens may be reevaluated)
|
||||
and generates all 128 tokens. Retain preparation evidence separately.
|
||||
Antirez DS4 uses a zero-output preparation request because generating a
|
||||
token advances its live cache past the prompt. Gufo DS4 first builds a C1
|
||||
prompt checkpoint, then prepares the concurrent sessions from it to keep
|
||||
prefill arithmetic consistent. Record both choices in the artifacts.
|
||||
llama.cpp's recurrent checkpoint is four tokens before the prompt frontier;
|
||||
other models may only need the final token for logits. This avoids mixing
|
||||
Gufo's active decode clock with llama.cpp's elapsed
|
||||
@@ -373,7 +402,7 @@ compared against the official reference transcript.
|
||||
Model: `qwen3-tts`. Reference: **audio.cpp** server (`qwen3_tts` model spec,
|
||||
ROCm), `/v1/audio/speech`, same text, seed, sampling and variant
|
||||
(CustomVoice / VoiceDesign / Base ICL); variants audio.cpp does not expose
|
||||
are `n/a`.
|
||||
are `N/A`.
|
||||
|
||||
1. **Loading.** Cold-file-cache readiness per variant; first-request and warm
|
||||
latency for a short sentence.
|
||||
@@ -441,7 +470,7 @@ explicit approval and `--allow-full-generation`.
|
||||
Render the tables, check that every Gufo/reference pair used the same
|
||||
workload identity, and keep only concise interpretation notes beside them. Put dates, method,
|
||||
server flags, artifact provenance and reproduction commands in `EVALUATION.md`.
|
||||
Remove statements the new numbers contradict. List the remaining `TODO` cells with the reason (tool missing,
|
||||
reference unsupported, time budget). Update `EXPERIMENTS.md` only when a measurement changes a
|
||||
Remove statements the new numbers contradict. Explain any remaining `TODO`
|
||||
measurements and `N/A` comparisons separately. Update `EXPERIMENTS.md` only when a measurement changes a
|
||||
retained decision. Summarize per table: Gufo, reference, best and worst gain,
|
||||
and every cell where completion hashes or transcripts did not match.
|
||||
@@ -0,0 +1,91 @@
|
||||
{
|
||||
lib,
|
||||
stdenv,
|
||||
fetchFromGitHub,
|
||||
rocmPackages,
|
||||
}:
|
||||
|
||||
let
|
||||
revision = "0aaea5a238fb41a35106a551e73c8409dfb751ac";
|
||||
headers = with rocmPackages; [
|
||||
hipblas
|
||||
hipblas-common
|
||||
hipblaslt
|
||||
rocblas
|
||||
rocwmma
|
||||
hipcub
|
||||
rocprim
|
||||
];
|
||||
libraries = with rocmPackages; [
|
||||
hipblas
|
||||
hipblaslt
|
||||
rocblas
|
||||
];
|
||||
in
|
||||
stdenv.mkDerivation (finalAttrs: {
|
||||
pname = "ds4-reference";
|
||||
version = "0-unstable-2026-09-20";
|
||||
|
||||
src = fetchFromGitHub {
|
||||
owner = "antirez";
|
||||
repo = "ds4";
|
||||
rev = revision;
|
||||
hash = "sha256-Bo/td1HwVjw6bwz3BDwTP+ZSudkVFCo2aIVK4axvmXg=";
|
||||
};
|
||||
|
||||
nativeBuildInputs = [ rocmPackages.clr ];
|
||||
buildInputs = [ rocmPackages.clr ] ++ headers;
|
||||
dontConfigure = true;
|
||||
|
||||
# Upstream's Strix Halo build, with explicit Nix paths and a reproducible
|
||||
# CPU target. No inference patches or build-time GPU detection.
|
||||
makeFlags = [
|
||||
"HIPCC=${rocmPackages.clr}/bin/hipcc"
|
||||
"ROCM_ARCH=gfx1151"
|
||||
"NATIVE_CPU_FLAG=-march=znver5"
|
||||
(
|
||||
"ROCM_CFLAGS=-O3 -ffast-math -g -fno-finite-math-only -pthread"
|
||||
+ " -D__HIP_PLATFORM_AMD__ -Wno-unused-command-line-argument"
|
||||
+ " --rocm-path=${rocmPackages.clr} --offload-arch=gfx1151"
|
||||
+ lib.concatMapStrings (p: " -I${lib.getDev p}/include") headers
|
||||
)
|
||||
(
|
||||
"ROCM_LDLIBS=-lm -pthread"
|
||||
+ lib.concatMapStrings (p: " -L${lib.getLib p}/lib") libraries
|
||||
+ " -lhipblas -lhipblaslt -lrocblas"
|
||||
)
|
||||
];
|
||||
|
||||
buildPhase = ''
|
||||
runHook preBuild
|
||||
make strix-halo -j"$NIX_BUILD_CORES" ${lib.escapeShellArgs finalAttrs.makeFlags}
|
||||
runHook postBuild
|
||||
'';
|
||||
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
install -Dm755 -t "$out/bin" ds4 ds4-server ds4-bench ds4-eval ds4-agent
|
||||
install -Dm644 LICENSE "$out/share/licenses/ds4-reference/LICENSE"
|
||||
cp -r licenses "$out/share/licenses/ds4-reference/third-party"
|
||||
printf '%s\n' '${revision}' > "$out/share/ds4-revision"
|
||||
runHook postInstall
|
||||
'';
|
||||
|
||||
doInstallCheck = true;
|
||||
installCheckPhase = ''
|
||||
runHook preInstallCheck
|
||||
"$out/bin/ds4-server" --help >/dev/null
|
||||
"$out/bin/ds4-bench" --help >/dev/null
|
||||
runHook postInstallCheck
|
||||
'';
|
||||
|
||||
passthru = { inherit revision; };
|
||||
|
||||
meta = {
|
||||
description = "Optional antirez/ds4 benchmark reference for Strix Halo";
|
||||
homepage = "https://github.com/antirez/ds4";
|
||||
license = lib.licenses.mit;
|
||||
platforms = [ "x86_64-linux" ];
|
||||
mainProgram = "ds4-server";
|
||||
};
|
||||
})
|
||||
@@ -0,0 +1,57 @@
|
||||
{
|
||||
lib,
|
||||
llama-cpp,
|
||||
fetchFromGitHub,
|
||||
rocmPackages,
|
||||
runCommand,
|
||||
}:
|
||||
let
|
||||
# b11069 includes Qwen3.8 and DFlash2. The separate MTP pin is the
|
||||
# Flash-Next implementation from ggml-org/llama.cpp#28243.
|
||||
revision = "68d9053afd4f4d0752ced6187585f862355a40be";
|
||||
mtpRevision = "6fcaa16f4b360649933a54d1f91ad40ed35c0e11";
|
||||
release =
|
||||
(llama-cpp.override {
|
||||
rocmSupport = true;
|
||||
rocmGpuTargets = [ "gfx1151" ];
|
||||
}).overrideAttrs
|
||||
(oldAttrs: {
|
||||
version = "11069";
|
||||
src = fetchFromGitHub {
|
||||
owner = "ggml-org";
|
||||
repo = "llama.cpp";
|
||||
rev = revision;
|
||||
hash = "sha256-BnGWYIkVe9y4aufhS5s3Jco1j/BY0MgBlzXrRQFMW3o=";
|
||||
};
|
||||
npmDepsHash = "sha256-2Q7XhaLAArmviOLdQsNbYTfdyDE5pW9lR26cRHEVl9k=";
|
||||
cmakeFlags = (oldAttrs.cmakeFlags or [ ]) ++ [
|
||||
"-DLLAMA_HIP_UMA=ON"
|
||||
"-DLLAMA_BUILD_COMMIT:STRING=${builtins.substring 0 8 revision}"
|
||||
];
|
||||
preConfigure = (oldAttrs.preConfigure or "") + ''
|
||||
cmakeFlagsArray+=("-DCMAKE_HIP_FLAGS=--rocm-path=${rocmPackages.clr} -mllvm --amdgpu-unroll-threshold-local=600")
|
||||
'';
|
||||
});
|
||||
mtpRuntime = release.overrideAttrs (oldAttrs: {
|
||||
pname = "llama-cpp-mtp";
|
||||
version = "11069";
|
||||
src = fetchFromGitHub {
|
||||
owner = "danielhanchen";
|
||||
repo = "llama.cpp";
|
||||
rev = mtpRevision;
|
||||
hash = "sha256-YgIkYHiV1LNA1OvTcB8SSdOeqr37+cEFAOQjEBfXcK4=";
|
||||
};
|
||||
cmakeFlags =
|
||||
builtins.filter (f: !(lib.hasPrefix "-DLLAMA_BUILD_COMMIT" f)) (oldAttrs.cmakeFlags or [ ])
|
||||
++ [ "-DLLAMA_BUILD_COMMIT:STRING=${builtins.substring 0 8 mtpRevision}" ];
|
||||
});
|
||||
in
|
||||
{
|
||||
inherit release;
|
||||
# Distinct executable names allow both reference revisions in one shell.
|
||||
mtp = runCommand "llama-cpp-mtp-bin" { } ''
|
||||
mkdir -p $out/bin
|
||||
ln -s ${mtpRuntime}/bin/llama-server $out/bin/llama-server-mtp
|
||||
ln -s ${mtpRuntime}/bin/llama-bench $out/bin/llama-bench-mtp
|
||||
'';
|
||||
}
|
||||
@@ -88,7 +88,10 @@ nix build
|
||||
|
||||
[flake.lock](flake.lock) pins the dependencies. `nix develop` adds profiling,
|
||||
model-download and independent evaluation tools; these are not runtime
|
||||
requirements. See [testing](docs/TESTING.md) for the small hosted CI suite and
|
||||
requirements. Optional benchmark baselines are selected separately with
|
||||
`nix shell .#ds4-reference`, `.#llama-cpp-reference` or
|
||||
`.#llama-cpp-mtp-reference`; see [benchmarking](docs/BENCHMARKS.md).
|
||||
See [testing](docs/TESTING.md) for the small hosted CI suite and
|
||||
explicit local quality checks.
|
||||
|
||||
### Without Nix
|
||||
|
||||
@@ -71,6 +71,8 @@ its upstream MIT notice. External model files are never part of the binary packa
|
||||
| NumPy / SciPy | Evaluation arrays and signal analysis | `BSD-3-Clause` | flake.lock | [NumPy](https://github.com/numpy/numpy), [SciPy](https://github.com/scipy/scipy) |
|
||||
| Requests | Evaluation HTTP client | `Apache-2.0` | flake.lock | [Requests](https://github.com/psf/requests) |
|
||||
| SoX | Development audio utility | `GPL-2.0-or-later` | flake.lock | [SoX](https://sourceforge.net/projects/sox/) |
|
||||
| antirez/ds4 | Optional benchmark package; not shipped with Gufo | `MIT` | `0aaea5a238fb41a35106a551e73c8409dfb751ac` | [Nix recipe](.devops/nix/ds4-reference.nix) |
|
||||
| llama.cpp reference builds | Optional benchmark packages; not shipped with Gufo | `MIT` | Release `68d9053a`; Flash-Next MTP `6fcaa16f` | [Nix recipes](.devops/nix/llama-cpp-reference.nix) |
|
||||
| Torchvision AlexNet weights | Evaluation data only; not shipped | `NOASSERTION` | SHA-256 `7be5be791159472b1fbf3c69796f7cb30dca7ad8466c2df70058c37116cdee02` | [PyTorch model distribution](https://download.pytorch.org/models/alexnet-owt-7be5be79.pth) |
|
||||
|
||||
Python/PyTorch and reference scripts are not installed with Gufo. Their own
|
||||
|
||||
+29
-3
@@ -14,11 +14,36 @@ results and qualification gaps live in:
|
||||
|
||||
`tools/bench/model-bench.py` measures the tables of a model's `BENCHMARKS.md`
|
||||
over HTTP for Gufo and for the reference project of its category (llama.cpp
|
||||
for GGUF language models), writes per-table artifacts, and renders the tables
|
||||
for Qwen; antirez/ds4 for DeepSeek), writes per-table artifacts, and renders the tables
|
||||
and SVG charts between the `<!-- bench:<id> -->` markers. Workloads and table
|
||||
layouts are declared in `docs/models/<model>/artifacts/bench.json`; the
|
||||
`benchmark-model` skill in `.agents/skills` describes the procedure.
|
||||
|
||||
Reference runtimes are explicitly selected Nix packages, excluded from Gufo,
|
||||
the normal development shell and hosted checks:
|
||||
|
||||
| Reference | Package | Executable |
|
||||
| --- | --- | --- |
|
||||
| Qwen AR / DFlash2 | `.#llama-cpp-reference` | `llama-server` |
|
||||
| Flash-Next MTP | `.#llama-cpp-mtp-reference` | `llama-server-mtp` |
|
||||
| DeepSeek AR / DSpark | `.#ds4-reference` | `ds4-server`, `ds4-bench` |
|
||||
|
||||
For example, enter the development shell and add only the needed baseline:
|
||||
|
||||
```sh
|
||||
nix develop
|
||||
nix shell .#llama-cpp-reference .#llama-cpp-mtp-reference
|
||||
python3 tools/bench/model-bench.py --model qwen3.8-flash-next --gguf "$MODEL" \
|
||||
--mtp "$MTP" run --target reference --table single-mtp
|
||||
```
|
||||
|
||||
DeepSeek's HTTP payload omits pp/tg durations. The driver reads the pinned
|
||||
server's existing stage timers from its log and checks them against HTTP token
|
||||
counts. Native `ds4-bench` also supports single-session AR/DSpark, but not
|
||||
concurrency. The pinned ROCm server disables DSpark when batching, so C>1
|
||||
DSpark comparisons are **N/A**. See the
|
||||
[DS4 method](models/deepseek-v4-flash/EVALUATION.md#benchmark-method).
|
||||
|
||||
New or refreshed text-model cards use the same pp2048 prose/copying prompts
|
||||
for single-user d0 and concurrency, with tg128. The driver shares their prompt
|
||||
generator and records prompt hashes. C1 should cross-check d0; investigate
|
||||
@@ -48,8 +73,9 @@ when prompt, cache state, sampling, speculation, and timed scope match.
|
||||
|
||||
Model tables report the **sum of individual request decode rates** in each
|
||||
concurrent group, averaged across measured groups. Prefill and queue time
|
||||
remain in the latency diagnostics. Missing decode timings are **TODO**;
|
||||
whole-request throughput is not a substitute.
|
||||
remain in the latency diagnostics. Pending measurements are **TODO**;
|
||||
comparisons prevented by missing reference features are **N/A**, with a reason.
|
||||
Whole-request throughput is not a substitute for decode timings.
|
||||
|
||||
`--endpoint-profile openai` supports other OpenAI-compatible servers. Missing
|
||||
server-stage metrics remain null. `--reference-report` compares completion
|
||||
|
||||
@@ -1,200 +1,110 @@
|
||||
# DeepSeek V4 Flash on Gufo
|
||||
# DeepSeek V4 Flash benchmarks
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| Host | Linux x86-64, AMD `gfx1151`, 128 GB unified memory; Nix release binaries |
|
||||
| Target | `antirez/deepseek-v4-gguf` revision `1cd7b564`: `DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf` (80.76 GiB) |
|
||||
| Speculative | DSpark support `DeepSeek-V4-Flash-DSpark-support-0731.gguf`, revision `e7f04037` |
|
||||
| Reference | llama.cpp `llama-server` release `b11069`, ROCm gfx1151 from `flake.nix`, same target GGUF and DSpark file through `--spec-type draft-dspark`; all llama.cpp cells are **TODO** until the first HTTP sweep |
|
||||
| Method | HTTP on both servers, same prompts and timed scope; single-user tables are **pp2048 / tg128** after a cached prefix of the stated depth; `C` is simultaneous requests |
|
||||
| Gain | Gufo over llama.cpp, positive when Gufo is faster |
|
||||
| Layout | [benchmark-model skill](../../../.agents/skills/benchmark-model/SKILL.md) |
|
||||
AMD Strix Halo `gfx1151`, 128 GB unified memory. Antirez Flash 0731 mixed
|
||||
`IQ2XXS` target, DSpark support sidecar. Greedy, thinking off.
|
||||
Reference: antirez/ds4 `0aaea5a2` (ROCm).
|
||||
|
||||
**Target parity remains open:** the optimized build misses the historical
|
||||
trajectory gate, and four post-prefill differential alerts remain unresolved.
|
||||
DSpark replay does not establish target correctness.
|
||||
[Evidence and limits](EVALUATION.md). Unknown current measurements are **TODO**.
|
||||
|
||||
## Loading
|
||||
|
||||
Cold-file-cache launch to HTTP readiness on 2026-09-21, including DSpark and
|
||||
two sessions at context capacity 4096. The integer expert-routing tensor
|
||||
loads without conversion; serving replay remains exact under the
|
||||
[state checks](EVALUATION.md).
|
||||
|
||||
<!-- bench:loading -->
|
||||
| Target | Gufo ready | llama.cpp ready | Gain |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| Flash 0731 | 36.82 s | TODO | TODO |
|
||||
<!-- /bench -->
|
||||
|
||||
|
||||
Measured 2026-09-23–24. One warmed sample per point. Positive gain favors Gufo.
|
||||
[Quality and measurement details](EVALUATION.md#benchmark-method).
|
||||
|
||||
## Single user, autoregressive
|
||||
|
||||
Latest depth control: **2026-09-19**, two release measurements at d0/d32K
|
||||
with `gufo bench`. Values are mean ± standard deviation; initial kernel
|
||||
setup contributes to depth-zero prefill variation. The final packing layout
|
||||
needs a d64K speed refresh. Gain uses means.
|
||||
Approximately pp2048 / tg128; depth is the cached prefix in tokens.
|
||||
|
||||
<!-- bench:single-ar -->
|
||||
| Depth | Gufo pp | llama.cpp pp | Gain | Gufo tg | llama.cpp tg | Gain |
|
||||
| DS4 Flash IQ2XXS AR<br>Depth (tokens) | Gufo pp (tok/s) | antirez/ds4 pp (tok/s) | Gain | Gufo tg (tok/s) | antirez/ds4 tg (tok/s) | Gain |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 0 | 450.26 ± 33.18 | TODO | TODO | 17.74 ± 0.00 | TODO | TODO |
|
||||
| 4,096 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8,192 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 12,288 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 16,384 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 32,768 | 422.18 ± 3.61 | TODO | TODO | 14.86 ± 0.00 | TODO | TODO |
|
||||
| 65,536 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 131,072 | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 0 | 484.62 | 248.41 | +95.1% | 16.28 | 14.83 | +9.8% |
|
||||
| 4,096 | 399.75 | 238.93 | +67.3% | 15.97 | 14.12 | +13.1% |
|
||||
| 8,192 | 368.17 | 223.24 | +64.9% | 15.82 | 13.90 | +13.8% |
|
||||
| 12,288 | 393.95 | 212.41 | +85.5% | 15.64 | 13.78 | +13.5% |
|
||||
| 16,384 | 358.41 | 214.97 | +66.7% | 15.50 | 13.64 | +13.6% |
|
||||
| 32,768 | 370.10 | 205.70 | +79.9% | 14.87 | 12.96 | +14.7% |
|
||||
| 65,536 | 321.28 | 177.43 | +81.1% | 13.95 | 12.00 | +16.2% |
|
||||
| 131,072 | 279.95 | 140.04 | +99.9% | 12.04 | 10.15 | +18.6% |
|
||||
<!-- /bench -->
|
||||
|
||||
|
||||
|
||||
## Single user, DSpark
|
||||
|
||||
Same workload and measurement date: two measurements at d0, one at d32K.
|
||||
DSpark prefill includes support-state work. The reference column is
|
||||
llama.cpp with the same DSpark support file (`draft-dspark`).
|
||||
pp is the highest measured rate per engine and depth across mixed/repetitive
|
||||
text.
|
||||
Antirez DSpark is N/A: its matched C1 control changes greedy output versus AR.
|
||||
|
||||
<!-- bench:single-dspark -->
|
||||
| Depth | Gufo pp | llama.cpp pp | Gain | Gufo tg | llama.cpp tg | Gain | Gufo accepted/step | llama.cpp accepted/step |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 0 | 453.49 ± 25.96 | TODO | TODO | 17.45 ± 0.00 | TODO | TODO | TODO | TODO |
|
||||
| 4,096 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8,192 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 12,288 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 16,384 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 32,768 | 417.80 | TODO | TODO | 35.30 | TODO | TODO | TODO | TODO |
|
||||
| 65,536 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 131,072 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| DS4 Flash IQ2XXS DSpark<br>Depth (tokens) | Gufo pp (tok/s) | antirez/ds4 pp (tok/s) | Gain pp | Gufo tg mixed (tok/s) | antirez/ds4 tg mixed (tok/s) | Gain mixed | Gufo tg repetitive (tok/s) | antirez/ds4 tg repetitive (tok/s) | Gain repetitive |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 0 | 483.11 | N/A | N/A | 18.25 | N/A | N/A | 26.62 | N/A | N/A |
|
||||
| 4,096 | 398.06 | N/A | N/A | 16.01 | N/A | N/A | 25.54 | N/A | N/A |
|
||||
| 8,192 | 367.84 | N/A | N/A | 16.48 | N/A | N/A | 25.03 | N/A | N/A |
|
||||
| 12,288 | 393.32 | N/A | N/A | 15.79 | N/A | N/A | 23.87 | N/A | N/A |
|
||||
| 16,384 | 358.90 | N/A | N/A | 15.52 | N/A | N/A | 23.81 | N/A | N/A |
|
||||
| 32,768 | 367.12 | N/A | N/A | 14.98 | N/A | N/A | 22.90 | N/A | N/A |
|
||||
| 65,536 | 336.67 | N/A | N/A | 14.05 | N/A | N/A | 19.76 | N/A | N/A |
|
||||
| 131,072 | 283.45 | N/A | N/A | 11.95 | N/A | N/A | 15.17 | N/A | N/A |
|
||||
<!-- /bench -->
|
||||
|
||||
|
||||
|
||||
Repetitive workload (**TODO**): same prefixes and depths, the measured turn
|
||||
asks the model to repeat the passage word for word, so the output is fully
|
||||
predictable — the single-user analogue of the `repetition` corpus below.
|
||||
## Multiple users, autoregressive
|
||||
|
||||
<!-- bench:single-dspark-repetition -->
|
||||
| Depth | Gufo pp | llama.cpp pp | Gain | Gufo tg | llama.cpp tg | Gain | Gufo accepted/step | llama.cpp accepted/step |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 0 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 4,096 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8,192 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 12,288 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 16,384 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 32,768 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 65,536 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 131,072 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
Same pp2048 prose prompt as single-user d0, tg128, context 4096 per user.
|
||||
All sessions prefilled before timed decoding; throughput sums individual rates.
|
||||
|
||||
<!-- bench:multi-ar -->
|
||||
| DS4 Flash IQ2XXS AR<br>Users | Gufo AR (tok/s) | antirez/ds4 AR (tok/s) | Gain |
|
||||
| ---: | ---: | ---: | ---: |
|
||||
| 1 | 16.27 | 14.81 | +9.9% |
|
||||
| 2 | 26.88 | 14.81 | +81.5% |
|
||||
| 4 | 41.09 | 14.89 | +176.0% |
|
||||
| 6 | 49.21 | 15.01 | +227.8% |
|
||||
| 8 | 55.47 | 14.99 | +270.0% |
|
||||
<!-- /bench -->
|
||||
|
||||
Thinking workload (**TODO**): thinking left on, the measured turn asks a
|
||||
question that requires reasoning.
|
||||
|
||||
|
||||
<!-- bench:single-dspark-thinking -->
|
||||
| Depth | Gufo pp | llama.cpp pp | Gain | Gufo tg | llama.cpp tg | Gain | Gufo accepted/step | llama.cpp accepted/step |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 0 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 4,096 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8,192 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 12,288 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 16,384 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 32,768 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 65,536 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 131,072 | TODO | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
## Multiple users, DSpark
|
||||
|
||||
Same pp2048 mixed/repetitive prompts as single-user d0, tg128, context 4096
|
||||
per user. All sessions prefilled before timed decoding; rates sum individual
|
||||
request decode rates. C1 cross-checks the single-user table.
|
||||
N/A: Antirez C1 DSpark fails the AR-equivalence control; ROCm batching disables
|
||||
DSpark at C>1. Gufo DSpark is not faster than AR in the largest cohorts here.
|
||||
|
||||
<!-- bench:multi-dspark -->
|
||||
| DS4 Flash IQ2XXS DSpark<br>Users | Gufo mixed (tok/s) | antirez/ds4 mixed (tok/s) | Gain | Gufo repetitive (tok/s) | antirez/ds4 repetitive (tok/s) | Gain |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | 18.24 | N/A | N/A | 26.59 | N/A | N/A |
|
||||
| 2 | 30.80 | N/A | N/A | 41.97 | N/A | N/A |
|
||||
| 4 | 41.20 | N/A | N/A | 47.04 | N/A | N/A |
|
||||
| 6 | 46.35 | N/A | N/A | 42.04 | N/A | N/A |
|
||||
| 8 | 54.74 | N/A | N/A | 52.77 | N/A | N/A |
|
||||
<!-- /bench -->
|
||||
|
||||
The CLI uses a repeating token sequence. Natural prompts can have substantially
|
||||
different acceptance and speed. Depth-zero generation starts from 16 tokens;
|
||||
it does not follow the measured 2048-token prefill.
|
||||
|
||||
|
||||
## Multiple users
|
||||
## Loading time
|
||||
|
||||
Aggregate delivered output tok/s (`aggregate.output_tokens_per_second.overall`).
|
||||
Context capacity 4096, greedy, 128 output tokens, `cache_prompt=false`, fresh
|
||||
server per point. Workloads come from the
|
||||
[speculative corpus](../qwen3.8-27b/artifacts/speculative-corpus.json):
|
||||
`repetition` runs `repetition_word`; `mixed` cycles through the distinct
|
||||
corpus cases. `Exact` counts llama.cpp completions whose hash matches the
|
||||
Gufo AR C1 reference. The current sweep is **TODO**, including acceptance and
|
||||
output checks at every concurrency.
|
||||
C1, context capacity 262144, DSpark. Cold target/sidecar files to HTTP readiness.
|
||||
|
||||
<!-- bench:multi-repetition -->
|
||||
| Users | Gufo AR | llama.cpp AR | Gain | Gufo DSpark | llama.cpp DSpark | Gain | Exact |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 2 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 4 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 6 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
<!-- bench:loading -->
|
||||
| DS4 Flash IQ2XXS<br>Target | Gufo ready (s) | antirez/ds4 ready (s) | Gain |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| IQ2XXS | 16.41 | 150.87 | +819.4% |
|
||||
<!-- /bench -->
|
||||
|
||||
<!-- bench:multi-mixed -->
|
||||
| Users | Gufo AR | llama.cpp AR | Gain | Gufo DSpark | llama.cpp DSpark | Gain | Exact |
|
||||
| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 2 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 4 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 6 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
| 8 | TODO | TODO | TODO | TODO | TODO | TODO | TODO |
|
||||
<!-- /bench -->
|
||||
|
||||
|
||||
Latest fixed-prompt sampled HTTP control (2026-09-19, C2, temperature 1,
|
||||
top-p 0.95, seed 7): **17.40 tok/s per user** for an autumn explanation,
|
||||
**21.01** for a repeating pattern, and **15.27** for a short naming task.
|
||||
Two fresh-server measurements; details and limits are in the
|
||||
[sampling evidence](EVALUATION.md#dspark).
|
||||
## Memory occupation
|
||||
|
||||
DSpark combines offline cycle costs and acceptance history with checkpoint
|
||||
confidence for filtered sampled C>1 requests. These use exact hybrid top-8
|
||||
proposals; C1 and unfiltered sampling retain point-mass proposals. Stochastic
|
||||
verification preserves the target distribution, with its own seeded trace.
|
||||
Greedy behavior is unchanged. Requests keep private RNG, proposal, acceptance
|
||||
and controller state; prefix reuse starts fresh request statistics. Live timing
|
||||
never changes token decisions.
|
||||
|
||||
## Memory
|
||||
|
||||
C1 with context capacity **262,144**. GPU-visible unified allocations exclude
|
||||
separate CPU memory; capacity does not imply filling the window. Gufo values
|
||||
come from the earlier C1 capacity control with DSpark (the first row was
|
||||
recorded as pp2048/4096 + tg128); a current refresh is TODO. llama.cpp resident device memory at the same `-c`: **TODO**.
|
||||
C1, context capacity 262144, AR. Peak memory reported by HIP.
|
||||
|
||||
<!-- bench:memory -->
|
||||
| Workload | Gufo GiB | llama.cpp GiB | Gain |
|
||||
| DS4 Flash IQ2XXS AR<br>Workload | Gufo GiB | antirez/ds4 GiB | Gain |
|
||||
| --- | ---: | ---: | ---: |
|
||||
| pp2048 + tg128 | 90.74 | TODO | TODO |
|
||||
| 16K prefix, pp4096 + tg128 | 91.46 | TODO | TODO |
|
||||
| pp2048 + tg128 | 93.09 | 99.96 | +7.4% |
|
||||
| 16K prefix, pp4096 + tg128 | 93.88 | 100.01 | +6.5% |
|
||||
<!-- /bench -->
|
||||
|
||||
|
||||
|
||||
Target and support weights, KV and working buffers all contribute. Compressed
|
||||
KV and score scratch grow with actual use. Prefill reuses scratch across
|
||||
ordered stages; indexer scoring borrows the idle attention-output range for
|
||||
temporary F16 keys. Queries share that range and feed WMMA directly.
|
||||
Persistent KV precision is unchanged.
|
||||
|
||||
## Reproduce
|
||||
|
||||
```sh
|
||||
nix build
|
||||
MODEL=/path/to/target.gguf
|
||||
DSPARK=/path/to/DSpark-support.gguf
|
||||
./result/bin/gufo bench --model "$MODEL" \
|
||||
-c 1 -p 2048 -n 128 -d 0,32768 -r 2 -v
|
||||
./result/bin/gufo bench --model "$MODEL" --dspark-model "$DSPARK" \
|
||||
-c 1 -p 2048 -n 128 -d 0,32768 -r 2 -v
|
||||
```
|
||||
|
||||
Use `-c 1,2,4,6,8` for concurrency and
|
||||
`-d 0,4096,8192,12288,16384,32768` for the full depth sweep. Use `-p 4096`
|
||||
for pp4096. Sampled controls use the same `--temperature` and `--seed` in both
|
||||
modes. Context preparation and restoration are outside timing. Verbose output
|
||||
records hashes and draft counts; compare these alongside speed. Run model
|
||||
jobs, builds and profiles sequentially. `gufo bench` is the kernel-iteration
|
||||
tool; the published comparison tables are measured over HTTP on both sides
|
||||
and rendered with `tools/bench/model-bench.py` as described by the
|
||||
`benchmark-model` skill. Distinguish cold requests from cache hits.
|
||||
|
||||
@@ -1,137 +1,68 @@
|
||||
# DeepSeek V4 Flash evaluation
|
||||
|
||||
Serving qualification (2026-09-21): AR/DSpark prefix and disk restoration,
|
||||
mixed sampling, EOS isolation, cancellation and late-arriving prefill passed.
|
||||
Snapshots retain the live attention/compression windows and committed DSpark
|
||||
frontier. The loader accepts the checkpoint's integer expert-mapping tensor
|
||||
without weakening tensor extent validation.
|
||||
**Target parity remains unresolved.** Antirez is an independent comparison,
|
||||
not official ground truth. AR/DSpark agreement cannot detect shared target errors.
|
||||
|
||||
The interrupted-chat check also passes on the production AR/DSpark server at
|
||||
262,144-token capacity: stop inside reasoning or visible text, retain or omit
|
||||
reasoning, replay greedy/seeded requests, continue a third turn, and restart
|
||||
from disk. Reasoning removal retains the stable prompt prefix; explicit
|
||||
`cache_prompt: false` reports no reuse. Run
|
||||
[`check-continuation.py`](../../../tools/serving/check-continuation.py) as
|
||||
described in [server documentation](../../SERVER.md). Exact replay uses the
|
||||
same prefill/decode history; it does not close the target arithmetic gaps below.
|
||||
## Quality
|
||||
|
||||
Focused C1 startup controls at the same capacity: target upload **14.49 s**,
|
||||
DSpark upload **0.56 s**. An empty digest cache required **46.72 s** for the
|
||||
target's full SHA-256 and **3.26 s** for DSpark; total load **65.55 s**.
|
||||
Both digests exactly match the previous full-file scans. Reused digests avoid
|
||||
that work. These are startup controls, not a refreshed performance sweep.
|
||||
The discarded scratch warmup before first prefill has been removed.
|
||||
|
||||
**Target parity is unresolved. No quality improvement is established by the
|
||||
current arithmetic audits.** Antirez's implementation is a differential control,
|
||||
not official ground truth. DSpark/AR agreement cannot detect shared target errors.
|
||||
|
||||
## Maintained checks
|
||||
|
||||
Tests live in `tests/models/deepseek_v4_flash`; commands in
|
||||
[tools/ds4](../../../tools/ds4/README.md). Run affected checks while iterating.
|
||||
|
||||
| Check | Required coverage |
|
||||
| Check | Retained result |
|
||||
| --- | --- |
|
||||
| `ds4.template`, `ds4.cli`, `ds4.dataset`, `ds4.eval` | Framing, wiring, pinned fixtures and probability/grading invariants |
|
||||
| `ds4.sampling` | Exact p/q and residual distributions, conditional proposals, penalties, seed replay and confidence stopping |
|
||||
| `ds4.projections` | Independent weight decoding, FP64 formulas, all IQ2 signs, MoE ownership, exact batch/scalar outputs, official HC/Sinkhorn equations |
|
||||
| `ds4.attention` | Independent attention, DSpark Markov/confidence and window formulas; exact prefill scores, masks, poisoned rows, ties, scratch bounds and top-k ordering |
|
||||
| `ds4.target` | Official tokens, pinned trajectory, full logits, replay, capacity equality and state isolation |
|
||||
| `ds4.dspark` | Exact scalar tokens/frontier logits through C8/16K, acceptance, policy, snapshots and forks |
|
||||
| `ds4.serving` | Sampling, physical batching, EOS checkpoint isolation, bounded prefill, prefix/disk caches, cancellation and exhaustion |
|
||||
| `ds4.chat` | Real sampled prompt and three-turn chat after length/EOS stops, AR/DSpark token identity and active drafting |
|
||||
| Optimized trajectory | **115/128** top-1 (required ≥116); rank sum 145 (≤142), worst rank 4 (≤3) |
|
||||
| Optimized versus Debug | **33/2327** greedy choices differ |
|
||||
| Post-prefill logits | **4/20** comparisons fail; post-decode vectors pass |
|
||||
| Reproducibility | **1280/1280** choices and 20 full-logit vectors repeat exactly |
|
||||
| Operators / state | FP64 formulas, 156 exact top-k cases and 736 DSpark replay choices pass |
|
||||
| [Sparse-indexer boundary](artifacts/indexer-boundary.json) | DSpark C1 and AR C2: 0/128 token differences, bit-identical full logits |
|
||||
| [Reference DSpark control](artifacts/reference-qualification.json) | Antirez C1 differs from its own AR after fresh and prepared prefill; performance comparison excluded |
|
||||
| Serving | Sampling, EOS isolation, cancellation, three-turn continuation and disk restore pass |
|
||||
| Capability | **53/75**, 22 failures, zero execution errors, nine length finishes; full refresh pending |
|
||||
|
||||
Evidence: [qualification](artifacts/quality-qualification.json),
|
||||
[prefill comparisons](artifacts/antirez-ds4-ar-comparison.json),
|
||||
[formulas](artifacts/prefill-formula-audit.json),
|
||||
[operators](artifacts/captured-operator-audit.json),
|
||||
[DSpark sampling](artifacts/dspark-sampling.json).
|
||||
DSpark uses p/q acceptance and residual correction; seeded replay requires the
|
||||
same execution configuration and schedule. The boundary fix does not resolve
|
||||
the separate target-parity alerts. Ask before rerunning the 75-question evaluation.
|
||||
|
||||
## Checks
|
||||
|
||||
[Maintained tools](../../../tools/ds4/README.md): `fast` checks wiring/sampling,
|
||||
`kernels` checks independent formulas, `model` checks logits and serving/state.
|
||||
Run affected checks while iterating; do not relax quality gates.
|
||||
|
||||
```sh
|
||||
nix develop -c tools/ds4/check.py fast
|
||||
nix develop -c tools/ds4/check.py kernels
|
||||
nix develop -c tools/ds4/check.py model --model "$MODEL" --dspark-model "$DSPARK"
|
||||
nix develop -c tools/ds4/check.py reference --model "$MODEL" \
|
||||
--upstream /path/to/pinned-antirez-checkout --output /tmp/ds4-reference
|
||||
```
|
||||
|
||||
The trajectory guard requires **116/128 top-1**, rank sum ≤142, worst rank ≤3.
|
||||
State controls require finite logits, **RMSE ≤1.12, cosine ≥0.979, max error ≤5**.
|
||||
Never relax limits to admit an optimization. Recent indexer changes preserve ten
|
||||
full-logit prefill vectors across 2K/4K calls and five depths, plus five vectors
|
||||
across a 64K disk snapshot, continued prefill and decode. Independent FP64
|
||||
operators, 156 exact top-k cases and 736 DSpark replay choices pass.
|
||||
## Benchmark method
|
||||
|
||||
## Target arithmetic
|
||||
Same Flash 0731 target/support GGUF, greedy, thinking off, pp2048/tg128;
|
||||
one warmed sample per point. Depth is cached prefix length. C1/2/4/6/8 use
|
||||
the same d0 prompts and prefill every session before timing decoding; throughput
|
||||
is the sum of individual request rates. Gufo uses a C1-built prompt checkpoint
|
||||
to hold prefill arithmetic fixed. These are decode-focused concurrency tests.
|
||||
At 64K, the mixed task requests a story because the summary stopped before tg128.
|
||||
Loading: C1/DSpark, capacity 262144, cold model files. Memory: C1/AR, same capacity,
|
||||
peak HIP-used memory. [Build/model identities](artifacts/model-identities.json).
|
||||
|
||||
The optimized 2026-09-19 build scores **115/128, rank sum 145, worst rank 4**;
|
||||
legacy Debug scores 116/128, 142, 3. The unchanged optimized baseline has
|
||||
identical logits. Mixed-library controls isolate build sensitivity to
|
||||
`backend.hip.cpp`; both use `-O3`, while `NDEBUG` changes assertion control flow
|
||||
in HIP shuffle wrappers. The precise cause remains TODO.
|
||||
Reference: [antirez/ds4 `0aaea5a2`](https://github.com/antirez/ds4/tree/0aaea5a238fb41a35106a551e73c8409dfb751ac),
|
||||
ROCm 7.2.3 / gfx1151, no inference patches. Gufo reports HTTP stage timings;
|
||||
Antirez's native log timers are checked against HTTP token counts. Unsupported
|
||||
comparisons are **N/A**, with reasons beside the tables. Antirez disables ROCm
|
||||
DSpark under native batching, including `--batched-session 1`; its C1 DSpark
|
||||
also fails the retained AR-equivalence control.
|
||||
|
||||
Hosted-checkpoint continuation likelihood covers 100 prompts / 2313 tokens,
|
||||
plus five smoke prompts / 14 tokens. API probabilities are saturated, so this
|
||||
is not full-distribution parity:
|
||||
```sh
|
||||
nix build .#ds4-reference --out-link result-ds4-reference
|
||||
nix develop -c python3 tools/bench/model-bench.py --model deepseek-v4-flash \
|
||||
--gguf "$MODEL" --dspark "$DSPARK" run --target gufo --todo
|
||||
# Reference: --target reference --reference-binary result-ds4-reference/bin/ds4-server
|
||||
```
|
||||
|
||||
| 100-case run | Average NLL ↓ |
|
||||
|
||||
Optimized/Debug disagree on **33/2327 greedy choices**. Their paired 100-case
|
||||
NLL delta is +0.001169 (95% case-bootstrap interval −0.003416 to +0.005655).
|
||||
Both retain 14/14 smoke tokens, but Gufo smoke NLL is 0.034343 versus 0.010908
|
||||
for the same-machine upstream control. This finite sample establishes neither
|
||||
equivalence nor an improvement.
|
||||
|
||||
[Matched prefill comparison](artifacts/antirez-ds4-ar-comparison.json): four of
|
||||
20 full-logit checks fail immediately after prefill; post-decode vectors pass.
|
||||
Gufo repeats exactly across 1280 choices and 20 vectors. The
|
||||
[formula audit](artifacts/prefill-formula-audit.json) retains all four alerts:
|
||||
|
||||
| Context / call size | RMSE | Cosine | Max logit error | Max probability difference |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 4K / 4K | 0.9691 | 0.98006 | 5.3386 | 5.90 percentage points |
|
||||
| 12K / 4K | 1.1819 | 0.96935 | 5.4680 | 39.81 percentage points |
|
||||
| 16K / 2K | 0.8128 | 0.98681 | 5.8521 | 11.63 percentage points |
|
||||
| 16K / 4K | 1.3476 | 0.96214 | 5.8032 | 9.80 percentage points |
|
||||
|
||||
The official Flash 0731 formula projects raw FP32 HC activations before RMS
|
||||
scaling. Gufo follows that order with F16 GGUF weights; the Antirez wide path
|
||||
normalizes first and rounds inputs to F16. Its own 12K probabilities change by
|
||||
up to 40.58 percentage points across call sizes, versus 6.00 for Gufo.
|
||||
Imitating that rounding fails the independent formula oracle and worsens two
|
||||
probability comparisons. It is rejected.
|
||||
|
||||
[Captured-input audit](artifacts/captured-operator-audit.json), layers 0/2/3/42:
|
||||
HC projection, Sinkhorn, reduction and normalization match independent equations;
|
||||
raw FP8 KV bytes and sampled selectors match. F16 router/compressor rounding can
|
||||
change nearly tied expert choices. FP32 compressor/router prototypes nevertheless
|
||||
fail end-to-end qualification and were removed. The 16K trace shows accumulated
|
||||
post-attention error (RMSE 0.000404 at layer 0, 0.994 at layer 42), without proving
|
||||
one root cause. Local precision alone is insufficient evidence of better output.
|
||||
|
||||
Next correction: controlled operator/layer ablation, captured-input regression,
|
||||
all four frontiers, reproducibility and official continuation checks. Full
|
||||
original-checkpoint distribution and capability qualification remain **TODO**.
|
||||
|
||||
Source pins: official `deepseek-ai/DeepSeek-V4-Flash-0731`
|
||||
`7872f01b1d1fe23eabc4c98b48bffcef5a386062`; differential `antirez/ds4`
|
||||
`6289c516273979173abbc062209a81dd3706b804`. Artifact hashes and methods remain
|
||||
in the retained audit JSONs.
|
||||
|
||||
## DSpark
|
||||
|
||||
Exact p/q acceptance and normalized residual correction preserve the target
|
||||
distribution. Filtered sampled C>1 uses hybrid top-8 proposals; C1/unfiltered
|
||||
sampling retains point-mass proposals. Offline costs, acceptance history and
|
||||
checkpoint confidence choose work; live timings never change token decisions.
|
||||
RNG, policy and acceptance state are request-private and reset on prefix reuse.
|
||||
Seeded replay requires the same execution configuration/schedule.
|
||||
|
||||
[Sampling evidence](artifacts/dspark-sampling.json) retains the 2026-09-19 C2
|
||||
controls at T=1, top-p 0.95, seed 7: 17.40/21.01/15.27 per-user tok/s for
|
||||
explanation/repetition/naming. [Controller costs](artifacts/cost-calibration.json)
|
||||
and [qualification summary](artifacts/quality-qualification.json) retain their
|
||||
own release identities; they are not a new whole-model qualification.
|
||||
|
||||
## Capability
|
||||
|
||||
The 2026-09-10 greedy C1 baseline passes **53/75** questions, with 22 failures,
|
||||
zero execution errors and nine length finishes (context 32768, output limit
|
||||
16000). A complete current comparison is **TODO**. The first 16 candidate cases
|
||||
repeat exactly: 13/16 versus baseline 14/16; the changed answer also changes in
|
||||
AR and matches DSpark. Request confirmation before the full 75-question rerun.
|
||||
The [benchmark driver](../../BENCHMARKS.md) retains counts, acceptance and commands
|
||||
in artifacts. The [optional reference package](../../../.devops/nix/ds4-reference.nix)
|
||||
is excluded from Gufo and the default dev shell; [build smoke evidence](artifacts/reference-smoke.json).
|
||||
@@ -3,18 +3,18 @@
|
||||
"category": "llm",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"docs": "deepseek-v4-flash"
|
||||
"docs": "deepseek-v4-flash",
|
||||
"label": "DS4 Flash IQ2XXS"
|
||||
},
|
||||
"files": {
|
||||
"gguf": "target GGUF (IQ2XXS mixed, antirez/deepseek-v4-gguf)",
|
||||
"dspark": "DSpark support GGUF"
|
||||
"gguf": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"dspark": "DeepSeek-V4-Flash-DSpark-support-0731.gguf"
|
||||
},
|
||||
"variants": {
|
||||
"default": {
|
||||
"label": "Flash 0731",
|
||||
"label": "IQ2XXS",
|
||||
"requires": [
|
||||
"gguf",
|
||||
"dspark"
|
||||
"gguf"
|
||||
]
|
||||
}
|
||||
},
|
||||
@@ -31,26 +31,13 @@
|
||||
],
|
||||
"reference": {
|
||||
"args": [
|
||||
"--spec-type",
|
||||
"draft-dspark",
|
||||
"--spec-draft-model",
|
||||
"{dspark}",
|
||||
"--spec-draft-ngl",
|
||||
"999"
|
||||
"--dspark",
|
||||
"--mtp-model",
|
||||
"{dspark}"
|
||||
]
|
||||
}
|
||||
},
|
||||
"tables": {
|
||||
"loading": {
|
||||
"title": "Cold-file-cache launch to readiness",
|
||||
"per_variant": false,
|
||||
"context": 4096,
|
||||
"sessions": 2,
|
||||
"repetitions": 1,
|
||||
"metric": "seconds",
|
||||
"better": "lower",
|
||||
"rows": "variants"
|
||||
},
|
||||
"single-ar": {
|
||||
"title": "Single user, autoregressive",
|
||||
"per_variant": false,
|
||||
@@ -74,7 +61,10 @@
|
||||
},
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher"
|
||||
"better": "higher",
|
||||
"depth_workloads": {
|
||||
"65536": "story"
|
||||
}
|
||||
},
|
||||
"single-dspark": {
|
||||
"title": "Single user, DSpark",
|
||||
@@ -101,72 +91,34 @@
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher",
|
||||
"workload": "prose"
|
||||
"workloads": {
|
||||
"single-dspark": {
|
||||
"label": "mixed",
|
||||
"workload": "prose",
|
||||
"depth_workloads": {
|
||||
"65536": "story"
|
||||
}
|
||||
},
|
||||
"single-dspark-repetition": {
|
||||
"label": "repetitive",
|
||||
"workload": "repetition",
|
||||
"depth_workloads": {
|
||||
"4096": "copy",
|
||||
"8192": "copy",
|
||||
"12288": "copy",
|
||||
"16384": "copy",
|
||||
"32768": "copy",
|
||||
"65536": "copy",
|
||||
"131072": "copy"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"single-dspark-repetition": {
|
||||
"title": "Single user, DSpark, repetitive",
|
||||
"per_variant": false,
|
||||
"speculative": true,
|
||||
"prompt_tokens": 2048,
|
||||
"output_tokens": 128,
|
||||
"depths": [
|
||||
0,
|
||||
4096,
|
||||
8192,
|
||||
12288,
|
||||
16384,
|
||||
32768,
|
||||
65536,
|
||||
131072
|
||||
],
|
||||
"context": 133760,
|
||||
"depth_tolerance": 0.005,
|
||||
"prefix": {
|
||||
"generator": "synthetic-paragraphs",
|
||||
"seed": 1
|
||||
},
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher",
|
||||
"workload": "repetition"
|
||||
},
|
||||
"single-dspark-thinking": {
|
||||
"title": "Single user, DSpark, thinking",
|
||||
"per_variant": false,
|
||||
"speculative": true,
|
||||
"prompt_tokens": 2048,
|
||||
"output_tokens": 128,
|
||||
"depths": [
|
||||
0,
|
||||
4096,
|
||||
8192,
|
||||
12288,
|
||||
16384,
|
||||
32768,
|
||||
65536,
|
||||
131072
|
||||
],
|
||||
"context": 133760,
|
||||
"depth_tolerance": 0.005,
|
||||
"prefix": {
|
||||
"generator": "synthetic-paragraphs",
|
||||
"seed": 1
|
||||
},
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher",
|
||||
"workload": "thinking"
|
||||
},
|
||||
"multi-repetition": {
|
||||
"title": "Multiple users, repetition",
|
||||
"multi-ar": {
|
||||
"title": "Multiple users, autoregressive",
|
||||
"per_variant": false,
|
||||
"modes": [
|
||||
"ar",
|
||||
"dspark"
|
||||
],
|
||||
"suite": "../../qwen3.8-27b/artifacts/speculative-corpus.json",
|
||||
"cases": [
|
||||
"repetition_word"
|
||||
"ar"
|
||||
],
|
||||
"corpus_layout": "homogeneous",
|
||||
"concurrency": [
|
||||
@@ -178,32 +130,21 @@
|
||||
],
|
||||
"context": 4096,
|
||||
"output_tokens": 128,
|
||||
"cache_prompt": false,
|
||||
"cache_prompt": true,
|
||||
"warmup": 1,
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher"
|
||||
"better": "higher",
|
||||
"prompt_tokens": 2048,
|
||||
"workload": "prose",
|
||||
"prefill_first": true
|
||||
},
|
||||
"multi-mixed": {
|
||||
"title": "Multiple users, mixed corpus",
|
||||
"multi-dspark": {
|
||||
"title": "Multiple users, DSpark",
|
||||
"per_variant": false,
|
||||
"modes": [
|
||||
"ar",
|
||||
"dspark"
|
||||
],
|
||||
"suite": "../../qwen3.8-27b/artifacts/speculative-corpus.json",
|
||||
"cases": [
|
||||
"expository_pangram",
|
||||
"cpp_ring_buffer",
|
||||
"reasoning_train",
|
||||
"summary_gpu",
|
||||
"italian_explanation",
|
||||
"chinese_explanation",
|
||||
"json_schedule",
|
||||
"creative_station",
|
||||
"instruction_debug"
|
||||
],
|
||||
"corpus_layout": "distinct",
|
||||
"concurrency": [
|
||||
1,
|
||||
2,
|
||||
@@ -213,14 +154,38 @@
|
||||
],
|
||||
"context": 4096,
|
||||
"output_tokens": 128,
|
||||
"cache_prompt": false,
|
||||
"cache_prompt": true,
|
||||
"warmup": 1,
|
||||
"repetitions": 1,
|
||||
"metric": "tok/s",
|
||||
"better": "higher"
|
||||
"better": "higher",
|
||||
"workloads": {
|
||||
"multi-mixed": {
|
||||
"label": "mixed",
|
||||
"corpus_layout": "homogeneous",
|
||||
"workload": "prose"
|
||||
},
|
||||
"multi-repetition": {
|
||||
"label": "repetitive",
|
||||
"corpus_layout": "homogeneous",
|
||||
"workload": "repetition"
|
||||
}
|
||||
},
|
||||
"prompt_tokens": 2048,
|
||||
"prefill_first": true
|
||||
},
|
||||
"loading": {
|
||||
"title": "Loading time",
|
||||
"per_variant": false,
|
||||
"context": 262144,
|
||||
"sessions": 1,
|
||||
"repetitions": 1,
|
||||
"metric": "seconds",
|
||||
"better": "lower",
|
||||
"rows": "variants"
|
||||
},
|
||||
"memory": {
|
||||
"title": "GPU-visible allocation",
|
||||
"title": "Memory occupation",
|
||||
"per_variant": false,
|
||||
"context": 262144,
|
||||
"workloads": [
|
||||
@@ -253,22 +218,15 @@
|
||||
"readiness": "/ready"
|
||||
},
|
||||
"reference": {
|
||||
"name": "llama.cpp",
|
||||
"server": "llama-server",
|
||||
"name": "antirez/ds4",
|
||||
"kind": "ds4",
|
||||
"server": "ds4-server",
|
||||
"revision": "0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"args": [
|
||||
"-ngl",
|
||||
"999",
|
||||
"-fa",
|
||||
"on",
|
||||
"--cache-reuse",
|
||||
"0",
|
||||
"--jinja",
|
||||
"--reasoning",
|
||||
"off",
|
||||
"--cache-ram",
|
||||
"0"
|
||||
"--rocm"
|
||||
],
|
||||
"endpoint_profile": "openai",
|
||||
"readiness": "/health"
|
||||
"readiness": "/v1/models",
|
||||
"stage_timings": true
|
||||
}
|
||||
}
|
||||
@@ -15,19 +15,19 @@
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 216
|
||||
L 288 216
|
||||
L 288 0
|
||||
L 0 0
|
||||
<path d="M 0 216
|
||||
L 288 216
|
||||
L 288 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 41.173203 190.279297
|
||||
L 278.28 190.279297
|
||||
L 278.28 29.440781
|
||||
L 41.173203 29.440781
|
||||
<path d="M 46.899453 190.28
|
||||
L 278.28 190.28
|
||||
L 278.28 29.440781
|
||||
L 46.899453 29.440781
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
@@ -35,162 +35,207 @@ z
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_1">
|
||||
<defs>
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="159.726602" y="190.279297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="162.589727" y="190.28" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_1">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="159.726602" y="204.117891" transform="rotate(-0 159.726602 204.117891)">Flash 0731</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="162.589727" y="204.117891" transform="rotate(-0 162.589727 204.117891)">IQ2XXS</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1">
|
||||
<g id="line2d_2">
|
||||
<path d="M 41.173203 190.279297
|
||||
L 278.28 190.279297
|
||||
" clip-path="url(#p6d90444083)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 190.28
|
||||
L 278.28 190.28
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_3">
|
||||
<defs>
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="190.279297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="190.28" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_2">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="193.698242" transform="rotate(-0 34.173203 193.698242)">0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="193.698945" transform="rotate(-0 39.899453 193.698945)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_4">
|
||||
<path d="M 41.173203 153.877315
|
||||
L 278.28 153.877315
|
||||
" clip-path="url(#p6d90444083)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 168.070037
|
||||
L 278.28 168.070037
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_5">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="153.877315" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="168.070037" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_3">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="157.296261" transform="rotate(-0 34.173203 157.296261)">10</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="171.488983" transform="rotate(-0 39.899453 171.488983)">25</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_6">
|
||||
<path d="M 41.173203 117.475334
|
||||
L 278.28 117.475334
|
||||
" clip-path="url(#p6d90444083)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 145.860075
|
||||
L 278.28 145.860075
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_7">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="117.475334" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="145.860075" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="120.894279" transform="rotate(-0 34.173203 120.894279)">20</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="149.27902" transform="rotate(-0 39.899453 149.27902)">50</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_8">
|
||||
<path d="M 41.173203 81.073352
|
||||
L 278.28 81.073352
|
||||
" clip-path="url(#p6d90444083)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 123.650112
|
||||
L 278.28 123.650112
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_9">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="81.073352" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="123.650112" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="84.492297" transform="rotate(-0 34.173203 84.492297)">30</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="127.069058" transform="rotate(-0 39.899453 127.069058)">75</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_10">
|
||||
<path d="M 41.173203 44.67137
|
||||
L 278.28 44.67137
|
||||
" clip-path="url(#p6d90444083)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 101.44015
|
||||
L 278.28 101.44015
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_11">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="44.67137" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="101.44015" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="48.090316" transform="rotate(-0 34.173203 48.090316)">40</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="104.859095" transform="rotate(-0 39.899453 104.859095)">100</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="109.860039" transform="rotate(-90 16.558594 109.860039)">seconds to ready</text>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_12">
|
||||
<path d="M 46.899453 79.230187
|
||||
L 278.28 79.230187
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_13">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="79.230187" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="82.649132" transform="rotate(-0 39.899453 82.649132)">125</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_14">
|
||||
<path d="M 46.899453 57.020224
|
||||
L 278.28 57.020224
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_15">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="57.020224" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="60.43917" transform="rotate(-0 39.899453 60.43917)">150</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_8">
|
||||
<g id="line2d_16">
|
||||
<path d="M 46.899453 34.810262
|
||||
L 278.28 34.810262
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_17">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="34.810262" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="38.229207" transform="rotate(-0 39.899453 38.229207)">175</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="109.860391" transform="rotate(-90 16.558594 109.860391)">seconds to ready</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
<path d="M 51.950785 190.279297
|
||||
L 154.05419 190.279297
|
||||
L 154.05419 56.247201
|
||||
L 51.950785 56.247201
|
||||
<path d="M 57.416751 190.28
|
||||
L 157.054307 190.28
|
||||
L 157.054307 175.701381
|
||||
L 57.416751 175.701381
|
||||
z
|
||||
" clip-path="url(#p6d90444083)" style="fill: #2a78d6"/>
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 165.399013 190.279297
|
||||
L 267.502418 190.279297
|
||||
L 267.502418 190.279297
|
||||
L 165.399013 190.279297
|
||||
<path d="M 168.125146 190.28
|
||||
L 267.762702 190.28
|
||||
L 267.762702 56.247318
|
||||
L 168.125146 56.247318
|
||||
z
|
||||
" clip-path="url(#p6d90444083)" style="fill: #eb6834"/>
|
||||
" clip-path="url(#pfc1a2f0250)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
<path d="M 41.173203 190.279297
|
||||
L 41.173203 29.440781
|
||||
<path d="M 46.899453 190.28
|
||||
L 46.899453 29.440781
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_6">
|
||||
<path d="M 41.173203 190.279297
|
||||
L 278.28 190.279297
|
||||
<path d="M 46.899453 190.28
|
||||
L 278.28 190.28
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="patch_7">
|
||||
<path d="M 200.946094 44.379375
|
||||
L 218.946094 44.379375
|
||||
L 218.946094 38.079375
|
||||
L 200.946094 38.079375
|
||||
<path d="M 194.809219 44.379375
|
||||
L 212.809219 44.379375
|
||||
L 212.809219 38.079375
|
||||
L 194.809219 38.079375
|
||||
z
|
||||
" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="226.146094" y="44.379375" transform="rotate(-0 226.146094 44.379375)">Gufo</text>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="220.009219" y="44.379375" transform="rotate(-0 220.009219 44.379375)">Gufo</text>
|
||||
</g>
|
||||
<g id="patch_8">
|
||||
<path d="M 200.946094 57.880078
|
||||
L 218.946094 57.880078
|
||||
L 218.946094 51.580078
|
||||
L 200.946094 51.580078
|
||||
<path d="M 194.809219 57.880078
|
||||
L 212.809219 57.880078
|
||||
L 212.809219 51.580078
|
||||
L 194.809219 51.580078
|
||||
z
|
||||
" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="226.146094" y="57.880078" transform="rotate(-0 226.146094 57.880078)">llama.cpp</text>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="220.009219" y="57.880078" transform="rotate(-0 220.009219 57.880078)">antirez/ds4</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="2.88" y="11.918437" transform="rotate(-0 2.88 11.918437)">Cold-file-cache launch to readiness</text>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="2.88" y="11.918437" transform="rotate(-0 2.88 11.918437)">DS4 Flash IQ2XXS · Loading time</text>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p6d90444083">
|
||||
<rect x="41.173203" y="29.440781" width="237.106797" height="160.838516"/>
|
||||
<clipPath id="pfc1a2f0250">
|
||||
<rect x="46.899453" y="29.440781" width="231.380547" height="160.839219"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 8.4 KiB After Width: | Height: | Size: 11 KiB |
@@ -15,19 +15,19 @@
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 216
|
||||
L 374.4 216
|
||||
L 374.4 0
|
||||
L 0 0
|
||||
<path d="M 0 216
|
||||
L 374.4 216
|
||||
L 374.4 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
L 364.68 29.440781
|
||||
L 46.899453 29.440781
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
L 364.68 32.843096
|
||||
L 46.899453 32.843096
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
@@ -35,8 +35,8 @@ z
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_1">
|
||||
<defs>
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
@@ -61,14 +61,14 @@ L 0 3.5
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1">
|
||||
<g id="line2d_3">
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_4">
|
||||
<defs>
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
@@ -81,157 +81,172 @@ L -3.5 0
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_5">
|
||||
<path d="M 46.899453 160.969851
|
||||
L 364.68 160.969851
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 164.042554
|
||||
L 364.68 164.042554
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_6">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="160.969851" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="164.042554" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="164.388796" transform="rotate(-0 39.899453 164.388796)">20</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="167.461499" transform="rotate(-0 39.899453 167.461499)">20</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_7">
|
||||
<path d="M 46.899453 131.660405
|
||||
L 364.68 131.660405
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 137.805811
|
||||
L 364.68 137.805811
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_8">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="131.660405" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="137.805811" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="135.07935" transform="rotate(-0 39.899453 135.07935)">40</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="141.224756" transform="rotate(-0 39.899453 141.224756)">40</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_9">
|
||||
<path d="M 46.899453 102.350959
|
||||
L 364.68 102.350959
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 111.569068
|
||||
L 364.68 111.569068
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_10">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="102.350959" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="111.569068" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="105.769904" transform="rotate(-0 39.899453 105.769904)">60</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="114.988013" transform="rotate(-0 39.899453 114.988013)">60</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_11">
|
||||
<path d="M 46.899453 73.041513
|
||||
L 364.68 73.041513
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 85.332324
|
||||
L 364.68 85.332324
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_12">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="73.041513" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="85.332324" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="76.460458" transform="rotate(-0 39.899453 76.460458)">80</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="88.75127" transform="rotate(-0 39.899453 88.75127)">80</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_13">
|
||||
<path d="M 46.899453 43.732067
|
||||
L 364.68 43.732067
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 59.095581
|
||||
L 364.68 59.095581
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_14">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="43.732067" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="59.095581" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="47.151012" transform="rotate(-0 39.899453 47.151012)">100</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="62.514527" transform="rotate(-0 39.899453 62.514527)">100</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="109.860039" transform="rotate(-90 16.558594 109.860039)">GiB</text>
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_15">
|
||||
<path d="M 46.899453 32.858838
|
||||
L 364.68 32.858838
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_16">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="32.858838" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="36.277784" transform="rotate(-0 39.899453 36.277784)">120</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="111.561197" transform="rotate(-90 16.558594 111.561197)">GiB</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
<path d="M 61.344023 190.279297
|
||||
L 120.435447 190.279297
|
||||
L 120.435447 57.302341
|
||||
L 61.344023 57.302341
|
||||
<path d="M 61.344023 190.279297
|
||||
L 120.435447 190.279297
|
||||
L 120.435447 68.160376
|
||||
L 61.344023 68.160376
|
||||
z
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: #2a78d6"/>
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 225.486868 190.279297
|
||||
L 284.578292 190.279297
|
||||
L 284.578292 56.247201
|
||||
L 225.486868 56.247201
|
||||
<path d="M 225.486868 190.279297
|
||||
L 284.578292 190.279297
|
||||
L 284.578292 67.124025
|
||||
L 225.486868 67.124025
|
||||
z
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: #2a78d6"/>
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
<path d="M 127.001161 190.279297
|
||||
L 186.092585 190.279297
|
||||
L 186.092585 190.279297
|
||||
L 127.001161 190.279297
|
||||
<path d="M 127.001161 190.279297
|
||||
L 186.092585 190.279297
|
||||
L 186.092585 59.148055
|
||||
L 127.001161 59.148055
|
||||
z
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: #eb6834"/>
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_6">
|
||||
<path d="M 291.144006 190.279297
|
||||
L 350.23543 190.279297
|
||||
L 350.23543 190.279297
|
||||
L 291.144006 190.279297
|
||||
<path d="M 291.144006 190.279297
|
||||
L 350.23543 190.279297
|
||||
L 350.23543 59.082463
|
||||
L 291.144006 59.082463
|
||||
z
|
||||
" clip-path="url(#pbd8cc678c1)" style="fill: #eb6834"/>
|
||||
" clip-path="url(#p6f2e8923cc)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_7">
|
||||
<path d="M 46.899453 190.279297
|
||||
L 46.899453 29.440781
|
||||
<path d="M 46.899453 190.279297
|
||||
L 46.899453 32.843096
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_8">
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
<path d="M 46.899453 190.279297
|
||||
L 364.68 190.279297
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="patch_9">
|
||||
<path d="M 287.346094 44.379375
|
||||
L 305.346094 44.379375
|
||||
L 305.346094 38.079375
|
||||
L 287.346094 38.079375
|
||||
<path d="M 281.209219 47.78169
|
||||
L 299.209219 47.78169
|
||||
L 299.209219 41.48169
|
||||
L 281.209219 41.48169
|
||||
z
|
||||
" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="312.546094" y="44.379375" transform="rotate(-0 312.546094 44.379375)">Gufo</text>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="306.409219" y="47.78169" transform="rotate(-0 306.409219 47.78169)">Gufo</text>
|
||||
</g>
|
||||
<g id="patch_10">
|
||||
<path d="M 287.346094 57.880078
|
||||
L 305.346094 57.880078
|
||||
L 305.346094 51.580078
|
||||
L 287.346094 51.580078
|
||||
<path d="M 281.209219 61.282393
|
||||
L 299.209219 61.282393
|
||||
L 299.209219 54.982393
|
||||
L 281.209219 54.982393
|
||||
z
|
||||
" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="312.546094" y="57.880078" transform="rotate(-0 312.546094 57.880078)">llama.cpp</text>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="306.409219" y="61.282393" transform="rotate(-0 306.409219 61.282393)">antirez/ds4</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="3.744" y="11.918437" transform="rotate(-0 3.744 11.918437)">GPU-visible allocation</text>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="3.744" y="11.918437" transform="rotate(-0 3.744 11.918437)">DS4 Flash IQ2XXS AR · Memory occupation</text>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="pbd8cc678c1">
|
||||
<rect x="46.899453" y="29.440781" width="317.780547" height="160.838516"/>
|
||||
<clipPath id="p6f2e8923cc">
|
||||
<rect x="46.899453" y="32.843096" width="317.780547" height="157.436201"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 10 KiB After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1,333 @@
|
||||
<?xml version="1.0" encoding="utf-8" standalone="no"?>
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
|
||||
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="468pt" height="230.4pt" viewBox="0 0 468 230.4" xmlns="http://www.w3.org/2000/svg" version="1.1">
|
||||
<metadata>
|
||||
<rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
|
||||
<cc:Work>
|
||||
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
|
||||
<dc:format>image/svg+xml</dc:format>
|
||||
</cc:Work>
|
||||
</rdf:RDF>
|
||||
</metadata>
|
||||
<defs>
|
||||
<style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 230.4
|
||||
L 468 230.4
|
||||
L 468 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 41.173203 191.68
|
||||
L 458.28 191.68
|
||||
L 458.28 29.440781
|
||||
L 41.173203 29.440781
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_1">
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_1">
|
||||
<defs>
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="90.403914" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_1">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="90.403914" y="205.517891" transform="rotate(-0 90.403914 205.517891)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_2">
|
||||
<g id="line2d_2">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="170.065258" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_2">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="170.065258" y="205.517891" transform="rotate(-0 170.065258 205.517891)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_3">
|
||||
<g id="line2d_3">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="249.726602" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_3">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="249.726602" y="205.517891" transform="rotate(-0 249.726602 205.517891)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_4">
|
||||
<g id="line2d_4">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="329.387946" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="329.387946" y="205.517891" transform="rotate(-0 329.387946 205.517891)">6</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_5">
|
||||
<g id="line2d_5">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="409.049289" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="409.049289" y="205.517891" transform="rotate(-0 409.049289 205.517891)">8</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="249.726602" y="218.517891" transform="rotate(-0 249.726602 218.517891)">concurrent users</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1">
|
||||
<g id="line2d_6">
|
||||
<path d="M 41.173203 191.68
|
||||
L 458.28 191.68
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_7">
|
||||
<defs>
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="195.098945" transform="rotate(-0 34.173203 195.098945)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_8">
|
||||
<path d="M 41.173203 167.306582
|
||||
L 458.28 167.306582
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_9">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="167.306582" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="170.725527" transform="rotate(-0 34.173203 170.725527)">10</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_10">
|
||||
<path d="M 41.173203 142.933164
|
||||
L 458.28 142.933164
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_11">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="142.933164" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="146.35211" transform="rotate(-0 34.173203 146.35211)">20</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_12">
|
||||
<path d="M 41.173203 118.559746
|
||||
L 458.28 118.559746
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_13">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="118.559746" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="121.978692" transform="rotate(-0 34.173203 121.978692)">30</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_14">
|
||||
<path d="M 41.173203 94.186328
|
||||
L 458.28 94.186328
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_15">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="94.186328" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="97.605274" transform="rotate(-0 34.173203 97.605274)">40</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_16">
|
||||
<path d="M 41.173203 69.812911
|
||||
L 458.28 69.812911
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_17">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="69.812911" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="73.231856" transform="rotate(-0 34.173203 73.231856)">50</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_18">
|
||||
<path d="M 41.173203 45.439493
|
||||
L 458.28 45.439493
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_19">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="45.439493" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="48.858438" transform="rotate(-0 34.173203 48.858438)">60</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_14">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="110.560391" transform="rotate(-90 16.558594 110.560391)">sum of request decode tok/s</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
<path d="M 60.132603 191.68
|
||||
L 88.810687 191.68
|
||||
L 88.810687 152.024449
|
||||
L 60.132603 152.024449
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 139.793947 191.68
|
||||
L 168.472031 191.68
|
||||
L 168.472031 126.164253
|
||||
L 139.793947 126.164253
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
<path d="M 219.455291 191.68
|
||||
L 248.133375 191.68
|
||||
L 248.133375 91.529626
|
||||
L 219.455291 91.529626
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_6">
|
||||
<path d="M 299.116635 191.68
|
||||
L 327.794719 191.68
|
||||
L 327.794719 71.738411
|
||||
L 299.116635 71.738411
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_7">
|
||||
<path d="M 378.777979 191.68
|
||||
L 407.456063 191.68
|
||||
L 407.456063 56.480651
|
||||
L 378.777979 56.480651
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="patch_8">
|
||||
<path d="M 91.997141 191.68
|
||||
L 120.675224 191.68
|
||||
L 120.675224 155.582968
|
||||
L 91.997141 155.582968
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_9">
|
||||
<path d="M 171.658485 191.68
|
||||
L 200.336568 191.68
|
||||
L 200.336568 155.582968
|
||||
L 171.658485 155.582968
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_10">
|
||||
<path d="M 251.319828 191.68
|
||||
L 279.997912 191.68
|
||||
L 279.997912 155.387981
|
||||
L 251.319828 155.387981
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_11">
|
||||
<path d="M 330.981172 191.68
|
||||
L 359.659256 191.68
|
||||
L 359.659256 155.0955
|
||||
L 330.981172 155.0955
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_12">
|
||||
<path d="M 410.642516 191.68
|
||||
L 439.3206 191.68
|
||||
L 439.3206 155.144247
|
||||
L 410.642516 155.144247
|
||||
z
|
||||
" clip-path="url(#p2d3f1ae1b2)" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="patch_13">
|
||||
<path d="M 41.173203 191.68
|
||||
L 41.173203 29.440781
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_14">
|
||||
<path d="M 41.173203 191.68
|
||||
L 458.28 191.68
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="patch_15">
|
||||
<path d="M 49.273203 44.379375
|
||||
L 67.273203 44.379375
|
||||
L 67.273203 38.079375
|
||||
L 49.273203 38.079375
|
||||
z
|
||||
" style="fill: #2a78d6"/>
|
||||
</g>
|
||||
<g id="text_15">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="74.473203" y="44.379375" transform="rotate(-0 74.473203 44.379375)">Gufo AR</text>
|
||||
</g>
|
||||
<g id="patch_16">
|
||||
<path d="M 49.273203 57.880078
|
||||
L 67.273203 57.880078
|
||||
L 67.273203 51.580078
|
||||
L 49.273203 51.580078
|
||||
z
|
||||
" style="fill: #eb6834"/>
|
||||
</g>
|
||||
<g id="text_16">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="74.473203" y="57.880078" transform="rotate(-0 74.473203 57.880078)">antirez/ds4 AR</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_17">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="4.68" y="12.206437" transform="rotate(-0 4.68 12.206437)">DS4 Flash IQ2XXS AR · Multiple users, autoregressive</text>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p2d3f1ae1b2">
|
||||
<rect x="41.173203" y="29.440781" width="417.106797" height="162.239219"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 14 KiB |
@@ -0,0 +1,601 @@
|
||||
<?xml version="1.0" encoding="utf-8" standalone="no"?>
|
||||
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
|
||||
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
|
||||
<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="648pt" height="230.4pt" viewBox="0 0 648 230.4" xmlns="http://www.w3.org/2000/svg" version="1.1">
|
||||
<metadata>
|
||||
<rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
|
||||
<cc:Work>
|
||||
<dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
|
||||
<dc:format>image/svg+xml</dc:format>
|
||||
</cc:Work>
|
||||
</rdf:RDF>
|
||||
</metadata>
|
||||
<defs>
|
||||
<style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 230.4
|
||||
L 648 230.4
|
||||
L 648 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 41.173203 191.68
|
||||
L 319.14 191.68
|
||||
L 319.14 43.647094
|
||||
L 41.173203 43.647094
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_1">
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_1">
|
||||
<defs>
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="75.832116" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_1">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="75.832116" y="205.517891" transform="rotate(-0 75.832116 205.517891)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_2">
|
||||
<g id="line2d_2">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="133.790163" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_2">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="133.790163" y="205.517891" transform="rotate(-0 133.790163 205.517891)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_3">
|
||||
<g id="line2d_3">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="191.748211" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_3">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="191.748211" y="205.517891" transform="rotate(-0 191.748211 205.517891)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_4">
|
||||
<g id="line2d_4">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="249.706259" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="249.706259" y="205.517891" transform="rotate(-0 249.706259 205.517891)">6</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_5">
|
||||
<g id="line2d_5">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="307.664307" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="307.664307" y="205.517891" transform="rotate(-0 307.664307 205.517891)">8</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="180.156602" y="218.517891" transform="rotate(-0 180.156602 218.517891)">concurrent users</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1">
|
||||
<g id="line2d_6">
|
||||
<path d="M 41.173203 191.68
|
||||
L 319.14 191.68
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_7">
|
||||
<defs>
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="195.098945" transform="rotate(-0 34.173203 195.098945)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_8">
|
||||
<path d="M 41.173203 169.144239
|
||||
L 319.14 169.144239
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_9">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="169.144239" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="172.563184" transform="rotate(-0 34.173203 172.563184)">10</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_10">
|
||||
<path d="M 41.173203 146.608478
|
||||
L 319.14 146.608478
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_11">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="146.608478" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="150.027423" transform="rotate(-0 34.173203 150.027423)">20</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_12">
|
||||
<path d="M 41.173203 124.072717
|
||||
L 319.14 124.072717
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_13">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="124.072717" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_10">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="127.491663" transform="rotate(-0 34.173203 127.491663)">30</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_14">
|
||||
<path d="M 41.173203 101.536956
|
||||
L 319.14 101.536956
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_15">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="101.536956" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="104.955902" transform="rotate(-0 34.173203 104.955902)">40</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_16">
|
||||
<path d="M 41.173203 79.001195
|
||||
L 319.14 79.001195
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_17">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="79.001195" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="82.420141" transform="rotate(-0 34.173203 82.420141)">50</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_18">
|
||||
<path d="M 41.173203 56.465435
|
||||
L 319.14 56.465435
|
||||
" clip-path="url(#p87799a34b1)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_19">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="41.173203" y="56.465435" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="34.173203" y="59.88438" transform="rotate(-0 34.173203 59.88438)">60</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_14">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="16.558594" y="117.663547" transform="rotate(-90 16.558594 117.663547)">sum of request decode tok/s</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_3">
|
||||
<path d="M 53.808058 191.68
|
||||
L 74.672955 191.68
|
||||
L 74.672955 150.574772
|
||||
L 53.808058 150.574772
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 111.766105 191.68
|
||||
L 132.631002 191.68
|
||||
L 132.631002 122.269856
|
||||
L 111.766105 122.269856
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_5">
|
||||
<path d="M 169.724153 191.68
|
||||
L 190.58905 191.68
|
||||
L 190.58905 98.832665
|
||||
L 169.724153 98.832665
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_6">
|
||||
<path d="M 227.682201 191.68
|
||||
L 248.547098 191.68
|
||||
L 248.547098 87.226748
|
||||
L 227.682201 87.226748
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_7">
|
||||
<path d="M 285.640248 191.68
|
||||
L 306.505146 191.68
|
||||
L 306.505146 68.319245
|
||||
L 285.640248 68.319245
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_8">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_9">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_10">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_11">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_12">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#p87799a34b1)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_13">
|
||||
<path d="M 41.173203 191.68
|
||||
L 41.173203 43.647094
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_14">
|
||||
<path d="M 41.173203 191.68
|
||||
L 319.14 191.68
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="text_15">
|
||||
<text style="font-size: 10.8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #0b0b0b" x="180.156602" y="37.647094" transform="rotate(-0 180.156602 37.647094)">Mixed</text>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="patch_15">
|
||||
<path d="M 49.273203 58.585688
|
||||
L 67.273203 58.585688
|
||||
L 67.273203 52.285688
|
||||
L 49.273203 52.285688
|
||||
z
|
||||
" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="text_16">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="74.473203" y="58.585688" transform="rotate(-0 74.473203 58.585688)">Gufo</text>
|
||||
</g>
|
||||
<g id="patch_16">
|
||||
<path d="M 49.273203 72.086391
|
||||
L 67.273203 72.086391
|
||||
L 67.273203 65.786391
|
||||
L 49.273203 65.786391
|
||||
z
|
||||
" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="text_17">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="74.473203" y="72.086391" transform="rotate(-0 74.473203 72.086391)">antirez/ds4</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="axes_2">
|
||||
<g id="patch_17">
|
||||
<path d="M 360.313203 191.68
|
||||
L 638.28 191.68
|
||||
L 638.28 43.647094
|
||||
L 360.313203 43.647094
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="matplotlib.axis_3">
|
||||
<g id="xtick_6">
|
||||
<g id="line2d_20">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="394.972116" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_18">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="394.972116" y="205.517891" transform="rotate(-0 394.972116 205.517891)">1</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_7">
|
||||
<g id="line2d_21">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="452.930163" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_19">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="452.930163" y="205.517891" transform="rotate(-0 452.930163 205.517891)">2</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_8">
|
||||
<g id="line2d_22">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="510.888211" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_20">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="510.888211" y="205.517891" transform="rotate(-0 510.888211 205.517891)">4</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_9">
|
||||
<g id="line2d_23">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="568.846259" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_21">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="568.846259" y="205.517891" transform="rotate(-0 568.846259 205.517891)">6</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_10">
|
||||
<g id="line2d_24">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="626.804307" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_22">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="626.804307" y="205.517891" transform="rotate(-0 626.804307 205.517891)">8</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_23">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="499.296602" y="218.517891" transform="rotate(-0 499.296602 218.517891)">concurrent users</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_4">
|
||||
<g id="ytick_8">
|
||||
<g id="line2d_25">
|
||||
<path d="M 360.313203 191.68
|
||||
L 638.28 191.68
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_26">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="191.68" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_24">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="195.098945" transform="rotate(-0 353.313203 195.098945)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_9">
|
||||
<g id="line2d_27">
|
||||
<path d="M 360.313203 168.302938
|
||||
L 638.28 168.302938
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_28">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="168.302938" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_25">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="171.721883" transform="rotate(-0 353.313203 171.721883)">10</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_10">
|
||||
<g id="line2d_29">
|
||||
<path d="M 360.313203 144.925876
|
||||
L 638.28 144.925876
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_30">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="144.925876" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_26">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="148.344822" transform="rotate(-0 353.313203 148.344822)">20</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_11">
|
||||
<g id="line2d_31">
|
||||
<path d="M 360.313203 121.548815
|
||||
L 638.28 121.548815
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_32">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="121.548815" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_27">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="124.96776" transform="rotate(-0 353.313203 124.96776)">30</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_12">
|
||||
<g id="line2d_33">
|
||||
<path d="M 360.313203 98.171753
|
||||
L 638.28 98.171753
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_34">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="98.171753" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_28">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="101.590698" transform="rotate(-0 353.313203 101.590698)">40</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_13">
|
||||
<g id="line2d_35">
|
||||
<path d="M 360.313203 74.794691
|
||||
L 638.28 74.794691
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_36">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="74.794691" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_29">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="78.213636" transform="rotate(-0 353.313203 78.213636)">50</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_14">
|
||||
<g id="line2d_37">
|
||||
<path d="M 360.313203 51.417629
|
||||
L 638.28 51.417629
|
||||
" clip-path="url(#pc43547f688)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_38">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="360.313203" y="51.417629" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_30">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="353.313203" y="54.836574" transform="rotate(-0 353.313203 54.836574)">60</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_31">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="335.698594" y="117.663547" transform="rotate(-90 335.698594 117.663547)">sum of request decode tok/s</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="patch_18">
|
||||
<path d="M 372.948058 191.68
|
||||
L 393.812955 191.68
|
||||
L 393.812955 129.520393
|
||||
L 372.948058 129.520393
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_19">
|
||||
<path d="M 430.906105 191.68
|
||||
L 451.771002 191.68
|
||||
L 451.771002 93.566472
|
||||
L 430.906105 93.566472
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_20">
|
||||
<path d="M 488.864153 191.68
|
||||
L 509.72905 191.68
|
||||
L 509.72905 81.714301
|
||||
L 488.864153 81.714301
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_21">
|
||||
<path d="M 546.822201 191.68
|
||||
L 567.687098 191.68
|
||||
L 567.687098 93.402832
|
||||
L 546.822201 93.402832
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_22">
|
||||
<path d="M 604.780248 191.68
|
||||
L 625.645146 191.68
|
||||
L 625.645146 68.319245
|
||||
L 604.780248 68.319245
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="patch_23">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_24">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_25">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_26">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_27">
|
||||
<path d="M 0 0
|
||||
z
|
||||
" clip-path="url(#pc43547f688)" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="patch_28">
|
||||
<path d="M 360.313203 191.68
|
||||
L 360.313203 43.647094
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_29">
|
||||
<path d="M 360.313203 191.68
|
||||
L 638.28 191.68
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="text_32">
|
||||
<text style="font-size: 10.8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #0b0b0b" x="499.296602" y="37.647094" transform="rotate(-0 499.296602 37.647094)">Repetitive</text>
|
||||
</g>
|
||||
<g id="legend_2">
|
||||
<g id="patch_30">
|
||||
<path d="M 368.413203 58.585688
|
||||
L 386.413203 58.585688
|
||||
L 386.413203 52.285688
|
||||
L 368.413203 52.285688
|
||||
z
|
||||
" style="fill: #1baf7a"/>
|
||||
</g>
|
||||
<g id="text_33">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="393.613203" y="58.585688" transform="rotate(-0 393.613203 58.585688)">Gufo</text>
|
||||
</g>
|
||||
<g id="patch_31">
|
||||
<path d="M 368.413203 72.086391
|
||||
L 386.413203 72.086391
|
||||
L 386.413203 65.786391
|
||||
L 368.413203 65.786391
|
||||
z
|
||||
" style="fill: #eda100"/>
|
||||
</g>
|
||||
<g id="text_34">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="393.613203" y="72.086391" transform="rotate(-0 393.613203 72.086391)">antirez/ds4</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_35">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="6.48" y="12.206437" transform="rotate(-0 6.48 12.206437)">DS4 Flash IQ2XXS DSpark · Multiple users, DSpark</text>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p87799a34b1">
|
||||
<rect x="41.173203" y="43.647094" width="277.966797" height="148.032906"/>
|
||||
</clipPath>
|
||||
<clipPath id="pc43547f688">
|
||||
<rect x="360.313203" y="43.647094" width="277.966797" height="148.032906"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 26 KiB |
@@ -15,19 +15,19 @@
|
||||
</defs>
|
||||
<g id="figure_1">
|
||||
<g id="patch_1">
|
||||
<path d="M 0 230.4
|
||||
L 576 230.4
|
||||
L 576 0
|
||||
L 0 0
|
||||
<path d="M 0 230.4
|
||||
L 576 230.4
|
||||
L 576 0
|
||||
L 0 0
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
<g id="axes_1">
|
||||
<g id="patch_2">
|
||||
<path d="M 46.899453 191.679297
|
||||
L 270.169453 191.679297
|
||||
L 270.169453 29.440781
|
||||
L 46.899453 29.440781
|
||||
<path d="M 46.899453 191.679297
|
||||
L 279.392594 191.679297
|
||||
L 279.392594 29.440781
|
||||
L 46.899453 29.440781
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
@@ -35,103 +35,103 @@ z
|
||||
<g id="xtick_1">
|
||||
<g id="line2d_1">
|
||||
<defs>
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
<path id="m1ebf051f68" d="M 0 0
|
||||
L 0 3.5
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="54.598419" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="57.467323" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_1">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="54.598419" y="205.517188" transform="rotate(-0 54.598419 205.517188)">0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="57.467323" y="205.517188" transform="rotate(-0 57.467323 205.517188)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_2">
|
||||
<g id="line2d_2">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="85.394281" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="87.661237" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_2">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="85.394281" y="205.517188" transform="rotate(-0 85.394281 205.517188)">4K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="87.661237" y="205.517188" transform="rotate(-0 87.661237 205.517188)">4K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_3">
|
||||
<g id="line2d_3">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="116.190143" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="117.855152" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_3">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="116.190143" y="205.517188" transform="rotate(-0 116.190143 205.517188)">8K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="117.855152" y="205.517188" transform="rotate(-0 117.855152 205.517188)">8K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_4">
|
||||
<g id="line2d_4">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="146.986005" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="148.049066" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_4">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="146.986005" y="205.517188" transform="rotate(-0 146.986005 205.517188)">12K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="148.049066" y="205.517188" transform="rotate(-0 148.049066 205.517188)">12K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_5">
|
||||
<g id="line2d_5">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="177.781867" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="178.242981" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_5">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="177.781867" y="205.517188" transform="rotate(-0 177.781867 205.517188)">16K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="178.242981" y="205.517188" transform="rotate(-0 178.242981 205.517188)">16K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_6">
|
||||
<g id="line2d_6">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="208.577729" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="208.436895" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_6">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="208.577729" y="205.517188" transform="rotate(-0 208.577729 205.517188)">32K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="208.436895" y="205.517188" transform="rotate(-0 208.436895 205.517188)">32K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_7">
|
||||
<g id="line2d_7">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="239.373591" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="238.630809" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_7">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="239.373591" y="205.517188" transform="rotate(-0 239.373591 205.517188)">64K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="238.630809" y="205.517188" transform="rotate(-0 238.630809 205.517188)">64K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_8">
|
||||
<g id="line2d_8">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="270.169453" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="268.824724" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_8">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="270.169453" y="205.517188" transform="rotate(-0 270.169453 205.517188)">128K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="268.824724" y="205.517188" transform="rotate(-0 268.824724 205.517188)">128K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_9">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="158.534453" y="218.517891" transform="rotate(-0 158.534453 218.517891)">context depth (tokens)</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="163.146023" y="218.517891" transform="rotate(-0 163.146023 218.517891)">context depth (tokens)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_2">
|
||||
<g id="ytick_1">
|
||||
<g id="line2d_9">
|
||||
<path d="M 46.899453 191.679297
|
||||
L 270.169453 191.679297
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 191.679297
|
||||
L 279.392594 191.679297
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_10">
|
||||
<defs>
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
<path id="m680e5b1e24" d="M 0 0
|
||||
L -3.5 0
|
||||
" style="stroke: #52514e; stroke-width: 0.8"/>
|
||||
</defs>
|
||||
<g>
|
||||
@@ -144,77 +144,77 @@ L -3.5 0
|
||||
</g>
|
||||
<g id="ytick_2">
|
||||
<g id="line2d_11">
|
||||
<path d="M 46.899453 161.652476
|
||||
L 270.169453 161.652476
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 163.781405
|
||||
L 279.392594 163.781405
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_12">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="161.652476" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="163.781405" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_11">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="165.071421" transform="rotate(-0 39.899453 165.071421)">100</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="167.20035" transform="rotate(-0 39.899453 167.20035)">100</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_3">
|
||||
<g id="line2d_13">
|
||||
<path d="M 46.899453 131.625655
|
||||
L 270.169453 131.625655
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 135.883513
|
||||
L 279.392594 135.883513
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_14">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="131.625655" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="135.883513" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_12">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="135.044601" transform="rotate(-0 39.899453 135.044601)">200</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="139.302459" transform="rotate(-0 39.899453 139.302459)">200</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_4">
|
||||
<g id="line2d_15">
|
||||
<path d="M 46.899453 101.598835
|
||||
L 270.169453 101.598835
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 107.985622
|
||||
L 279.392594 107.985622
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_16">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="101.598835" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="107.985622" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_13">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="105.01778" transform="rotate(-0 39.899453 105.01778)">300</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="111.404567" transform="rotate(-0 39.899453 111.404567)">300</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_5">
|
||||
<g id="line2d_17">
|
||||
<path d="M 46.899453 71.572014
|
||||
L 270.169453 71.572014
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 80.08773
|
||||
L 279.392594 80.08773
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_18">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="71.572014" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="80.08773" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_14">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="74.990959" transform="rotate(-0 39.899453 74.990959)">400</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="83.506675" transform="rotate(-0 39.899453 83.506675)">400</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_6">
|
||||
<g id="line2d_19">
|
||||
<path d="M 46.899453 41.545193
|
||||
L 270.169453 41.545193
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 46.899453 52.189838
|
||||
L 279.392594 52.189838
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_20">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="41.545193" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="46.899453" y="52.189838" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_15">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="44.964139" transform="rotate(-0 39.899453 44.964139)">500</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="39.899453" y="55.608783" transform="rotate(-0 39.899453 55.608783)">500</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_16">
|
||||
@@ -222,43 +222,88 @@ L 270.169453 41.545193
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_21">
|
||||
<path d="M 54.598419 56.480534
|
||||
L 208.577729 64.912065
|
||||
" clip-path="url(#p587030b0bd)" style="fill: none; stroke: #2a78d6; stroke-width: 2; stroke-linecap: square"/>
|
||||
<path d="M 57.467323 56.480534
|
||||
L 87.661237 80.157475
|
||||
L 117.855152 88.967629
|
||||
L 148.049066 81.775552
|
||||
L 178.242981 91.690463
|
||||
L 208.436895 88.429199
|
||||
L 238.630809 102.04895
|
||||
L 268.824724 113.579149
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #2a78d6; stroke-width: 2; stroke-linecap: square"/>
|
||||
<defs>
|
||||
<path id="m91930ba968" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
C 1.55874 -2.683901 0.795609 -3 0 -3
|
||||
C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132
|
||||
C -2.683901 -1.55874 -3 -0.795609 -3 0
|
||||
C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132
|
||||
C -1.55874 2.683901 -0.795609 3 0 3
|
||||
<path id="m91930ba968" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
C 1.55874 -2.683901 0.795609 -3 0 -3
|
||||
C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132
|
||||
C -2.683901 -1.55874 -3 -0.795609 -3 0
|
||||
C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132
|
||||
C -1.55874 2.683901 -0.795609 3 0 3
|
||||
z
|
||||
" style="stroke: #fcfcfb"/>
|
||||
</defs>
|
||||
<g clip-path="url(#p587030b0bd)">
|
||||
<use xlink:href="#m91930ba968" x="54.598419" y="56.480534" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="208.577729" y="64.912065" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<g clip-path="url(#pe9847d3351)">
|
||||
<use xlink:href="#m91930ba968" x="57.467323" y="56.480534" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="87.661237" y="80.157475" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="117.855152" y="88.967629" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="148.049066" y="81.775552" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="178.242981" y="91.690463" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="208.436895" y="88.429199" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="238.630809" y="102.04895" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="268.824724" y="113.579149" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_22">
|
||||
<path d="M 57.467323 122.378144
|
||||
L 87.661237 125.022864
|
||||
L 117.855152 129.400043
|
||||
L 148.049066 132.421385
|
||||
L 178.242981 131.707199
|
||||
L 208.436895 134.293334
|
||||
L 238.630809 142.180068
|
||||
L 268.824724 152.611089
|
||||
" clip-path="url(#pe9847d3351)" style="fill: none; stroke: #eb6834; stroke-width: 2; stroke-linecap: square"/>
|
||||
<defs>
|
||||
<path id="ma0614988b8" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
C 1.55874 -2.683901 0.795609 -3 0 -3
|
||||
C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132
|
||||
C -2.683901 -1.55874 -3 -0.795609 -3 0
|
||||
C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132
|
||||
C -1.55874 2.683901 -0.795609 3 0 3
|
||||
z
|
||||
" style="stroke: #fcfcfb"/>
|
||||
</defs>
|
||||
<g clip-path="url(#pe9847d3351)">
|
||||
<use xlink:href="#ma0614988b8" x="57.467323" y="122.378144" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="87.661237" y="125.022864" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="117.855152" y="129.400043" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="148.049066" y="132.421385" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="178.242981" y="131.707199" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="208.436895" y="134.293334" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="238.630809" y="142.180068" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="268.824724" y="152.611089" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_22"/>
|
||||
<g id="patch_3">
|
||||
<path d="M 46.899453 191.679297
|
||||
L 46.899453 29.440781
|
||||
<path d="M 46.899453 191.679297
|
||||
L 46.899453 29.440781
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_4">
|
||||
<path d="M 46.899453 191.679297
|
||||
L 270.169453 191.679297
|
||||
<path d="M 46.899453 191.679297
|
||||
L 279.392594 191.679297
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="legend_1">
|
||||
<g id="line2d_23">
|
||||
<path d="M 54.999453 164.766484
|
||||
L 63.999453 164.766484
|
||||
L 72.999453 164.766484
|
||||
<path d="M 54.999453 164.766484
|
||||
L 63.999453 164.766484
|
||||
L 72.999453 164.766484
|
||||
" style="fill: none; stroke: #2a78d6; stroke-width: 2; stroke-linecap: square"/>
|
||||
<g>
|
||||
<use xlink:href="#m91930ba968" x="63.999453" y="164.766484" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
@@ -268,38 +313,25 @@ L 72.999453 164.766484
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="80.199453" y="167.916484" transform="rotate(-0 80.199453 167.916484)">Gufo</text>
|
||||
</g>
|
||||
<g id="line2d_24">
|
||||
<path d="M 54.999453 178.267188
|
||||
L 63.999453 178.267188
|
||||
L 72.999453 178.267188
|
||||
<path d="M 54.999453 178.267188
|
||||
L 63.999453 178.267188
|
||||
L 72.999453 178.267188
|
||||
" style="fill: none; stroke: #eb6834; stroke-width: 2; stroke-linecap: square"/>
|
||||
<defs>
|
||||
<path id="ma0614988b8" d="M 0 3
|
||||
C 0.795609 3 1.55874 2.683901 2.12132 2.12132
|
||||
C 2.683901 1.55874 3 0.795609 3 0
|
||||
C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132
|
||||
C 1.55874 -2.683901 0.795609 -3 0 -3
|
||||
C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132
|
||||
C -2.683901 -1.55874 -3 -0.795609 -3 0
|
||||
C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132
|
||||
C -1.55874 2.683901 -0.795609 3 0 3
|
||||
z
|
||||
" style="stroke: #fcfcfb"/>
|
||||
</defs>
|
||||
<g>
|
||||
<use xlink:href="#ma0614988b8" x="63.999453" y="178.267188" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_18">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="80.199453" y="181.417188" transform="rotate(-0 80.199453 181.417188)">llama.cpp</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="80.199453" y="181.417188" transform="rotate(-0 80.199453 181.417188)">antirez/ds4</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g id="axes_2">
|
||||
<g id="patch_5">
|
||||
<path d="M 331.469609 191.679297
|
||||
L 554.739609 191.679297
|
||||
L 554.739609 29.440781
|
||||
L 331.469609 29.440781
|
||||
<path d="M 331.469609 191.679297
|
||||
L 563.96275 191.679297
|
||||
L 563.96275 29.440781
|
||||
L 331.469609 29.440781
|
||||
z
|
||||
" style="fill: #fcfcfb"/>
|
||||
</g>
|
||||
@@ -307,93 +339,93 @@ z
|
||||
<g id="xtick_9">
|
||||
<g id="line2d_25">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="339.168575" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="342.037479" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_19">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="339.168575" y="205.517188" transform="rotate(-0 339.168575 205.517188)">0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="342.037479" y="205.517188" transform="rotate(-0 342.037479 205.517188)">0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_10">
|
||||
<g id="line2d_26">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="369.964437" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="372.231394" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_20">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="369.964437" y="205.517188" transform="rotate(-0 369.964437 205.517188)">4K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="372.231394" y="205.517188" transform="rotate(-0 372.231394 205.517188)">4K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_11">
|
||||
<g id="line2d_27">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="400.760299" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="402.425308" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_21">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="400.760299" y="205.517188" transform="rotate(-0 400.760299 205.517188)">8K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="402.425308" y="205.517188" transform="rotate(-0 402.425308 205.517188)">8K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_12">
|
||||
<g id="line2d_28">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="431.556161" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="432.619222" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_22">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="431.556161" y="205.517188" transform="rotate(-0 431.556161 205.517188)">12K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="432.619222" y="205.517188" transform="rotate(-0 432.619222 205.517188)">12K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_13">
|
||||
<g id="line2d_29">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="462.352023" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="462.813137" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_23">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="462.352023" y="205.517188" transform="rotate(-0 462.352023 205.517188)">16K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="462.813137" y="205.517188" transform="rotate(-0 462.813137 205.517188)">16K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_14">
|
||||
<g id="line2d_30">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="493.147885" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="493.007051" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_24">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="493.147885" y="205.517188" transform="rotate(-0 493.147885 205.517188)">32K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="493.007051" y="205.517188" transform="rotate(-0 493.007051 205.517188)">32K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_15">
|
||||
<g id="line2d_31">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="523.943747" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="523.200965" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_25">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="523.943747" y="205.517188" transform="rotate(-0 523.943747 205.517188)">64K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="523.200965" y="205.517188" transform="rotate(-0 523.200965 205.517188)">64K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="xtick_16">
|
||||
<g id="line2d_32">
|
||||
<g>
|
||||
<use xlink:href="#m1ebf051f68" x="554.739609" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m1ebf051f68" x="553.39488" y="191.679297" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_26">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="554.739609" y="205.517188" transform="rotate(-0 554.739609 205.517188)">128K</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="553.39488" y="205.517188" transform="rotate(-0 553.39488 205.517188)">128K</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_27">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="443.104609" y="218.517891" transform="rotate(-0 443.104609 218.517891)">context depth (tokens)</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="447.71618" y="218.517891" transform="rotate(-0 447.71618 218.517891)">context depth (tokens)</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="matplotlib.axis_4">
|
||||
<g id="ytick_7">
|
||||
<g id="line2d_33">
|
||||
<path d="M 331.469609 191.679297
|
||||
L 554.739609 191.679297
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 191.679297
|
||||
L 563.96275 191.679297
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_34">
|
||||
<g>
|
||||
@@ -406,159 +438,176 @@ L 554.739609 191.679297
|
||||
</g>
|
||||
<g id="ytick_8">
|
||||
<g id="line2d_35">
|
||||
<path d="M 331.469609 172.626484
|
||||
L 554.739609 172.626484
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 170.917816
|
||||
L 563.96275 170.917816
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_36">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="172.626484" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="170.917816" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_29">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="176.045429" transform="rotate(-0 324.469609 176.045429)">2.5</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="174.336761" transform="rotate(-0 324.469609 174.336761)">2.5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_9">
|
||||
<g id="line2d_37">
|
||||
<path d="M 331.469609 153.57367
|
||||
L 554.739609 153.57367
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 150.156335
|
||||
L 563.96275 150.156335
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_38">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="153.57367" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="150.156335" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_30">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="156.992616" transform="rotate(-0 324.469609 156.992616)">5.0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="153.575281" transform="rotate(-0 324.469609 153.575281)">5.0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_10">
|
||||
<g id="line2d_39">
|
||||
<path d="M 331.469609 134.520857
|
||||
L 554.739609 134.520857
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 129.394854
|
||||
L 563.96275 129.394854
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_40">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="134.520857" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="129.394854" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_31">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="137.939802" transform="rotate(-0 324.469609 137.939802)">7.5</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="132.8138" transform="rotate(-0 324.469609 132.8138)">7.5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_11">
|
||||
<g id="line2d_41">
|
||||
<path d="M 331.469609 115.468044
|
||||
L 554.739609 115.468044
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 108.633374
|
||||
L 563.96275 108.633374
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_42">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="115.468044" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="108.633374" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_32">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="118.886989" transform="rotate(-0 324.469609 118.886989)">10.0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="112.052319" transform="rotate(-0 324.469609 112.052319)">10.0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_12">
|
||||
<g id="line2d_43">
|
||||
<path d="M 331.469609 96.41523
|
||||
L 554.739609 96.41523
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 87.871893
|
||||
L 563.96275 87.871893
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_44">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="96.41523" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="87.871893" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_33">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="99.834176" transform="rotate(-0 324.469609 99.834176)">12.5</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="91.290838" transform="rotate(-0 324.469609 91.290838)">12.5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_13">
|
||||
<g id="line2d_45">
|
||||
<path d="M 331.469609 77.362417
|
||||
L 554.739609 77.362417
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 67.110412
|
||||
L 563.96275 67.110412
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_46">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="77.362417" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="67.110412" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_34">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="80.781363" transform="rotate(-0 324.469609 80.781363)">15.0</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="70.529357" transform="rotate(-0 324.469609 70.529357)">15.0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_14">
|
||||
<g id="line2d_47">
|
||||
<path d="M 331.469609 58.309604
|
||||
L 554.739609 58.309604
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
<path d="M 331.469609 46.348931
|
||||
L 563.96275 46.348931
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_48">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="58.309604" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="46.348931" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_35">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="61.728549" transform="rotate(-0 324.469609 61.728549)">17.5</text>
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="49.767877" transform="rotate(-0 324.469609 49.767877)">17.5</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="ytick_15">
|
||||
<g id="line2d_49">
|
||||
<path d="M 331.469609 39.256791
|
||||
L 554.739609 39.256791
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #e6e5e1; stroke-width: 0.6; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="line2d_50">
|
||||
<g>
|
||||
<use xlink:href="#m680e5b1e24" x="331.469609" y="39.256791" style="fill: #52514e; stroke: #52514e; stroke-width: 0.8"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_36">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end; fill: #52514e" x="324.469609" y="42.675736" transform="rotate(-0 324.469609 42.675736)">20.0</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_37">
|
||||
<g id="text_36">
|
||||
<text style="font-size: 9px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle; fill: #52514e" x="298.268438" y="110.560039" transform="rotate(-90 298.268438 110.560039)">generation tok/s</text>
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_51">
|
||||
<path d="M 339.168575 56.480534
|
||||
L 493.147885 78.429375
|
||||
" clip-path="url(#pc080eae31e)" style="fill: none; stroke: #2a78d6; stroke-width: 2; stroke-linecap: square"/>
|
||||
<g clip-path="url(#pc080eae31e)">
|
||||
<use xlink:href="#m91930ba968" x="339.168575" y="56.480534" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="493.147885" y="78.429375" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<g id="line2d_49">
|
||||
<path d="M 342.037479 56.480534
|
||||
L 372.231394 59.054957
|
||||
L 402.425308 60.300646
|
||||
L 432.619222 61.795473
|
||||
L 462.813137 62.958116
|
||||
L 493.007051 68.190009
|
||||
L 523.200965 75.830234
|
||||
L 553.39488 91.692005
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #2a78d6; stroke-width: 2; stroke-linecap: square"/>
|
||||
<g clip-path="url(#p9e1df01069)">
|
||||
<use xlink:href="#m91930ba968" x="342.037479" y="56.480534" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="372.231394" y="59.054957" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="402.425308" y="60.300646" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="432.619222" y="61.795473" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="462.813137" y="62.958116" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="493.007051" y="68.190009" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="523.200965" y="75.830234" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#m91930ba968" x="553.39488" y="91.692005" style="fill: #2a78d6; stroke: #fcfcfb"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_50">
|
||||
<path d="M 342.037479 68.522193
|
||||
L 372.231394 74.418453
|
||||
L 402.425308 76.245464
|
||||
L 432.619222 77.242015
|
||||
L 462.813137 78.404658
|
||||
L 493.007051 84.05178
|
||||
L 523.200965 92.024189
|
||||
L 553.39488 107.387685
|
||||
" clip-path="url(#p9e1df01069)" style="fill: none; stroke: #eb6834; stroke-width: 2; stroke-linecap: square"/>
|
||||
<g clip-path="url(#p9e1df01069)">
|
||||
<use xlink:href="#ma0614988b8" x="342.037479" y="68.522193" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="372.231394" y="74.418453" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="402.425308" y="76.245464" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="432.619222" y="77.242015" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="462.813137" y="78.404658" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="493.007051" y="84.05178" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="523.200965" y="92.024189" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
<use xlink:href="#ma0614988b8" x="553.39488" y="107.387685" style="fill: #eb6834; stroke: #fcfcfb"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="line2d_52"/>
|
||||
<g id="patch_6">
|
||||
<path d="M 331.469609 191.679297
|
||||
L 331.469609 29.440781
|
||||
<path d="M 331.469609 191.679297
|
||||
L 331.469609 29.440781
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
<g id="patch_7">
|
||||
<path d="M 331.469609 191.679297
|
||||
L 554.739609 191.679297
|
||||
<path d="M 331.469609 191.679297
|
||||
L 563.96275 191.679297
|
||||
" style="fill: none; stroke: #e6e5e1; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
|
||||
</g>
|
||||
</g>
|
||||
<g id="text_38">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="5.76" y="12.206437" transform="rotate(-0 5.76 12.206437)">Single user, autoregressive</text>
|
||||
<g id="text_37">
|
||||
<text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start; fill: #0b0b0b" x="5.76" y="12.206437" transform="rotate(-0 5.76 12.206437)">DS4 Flash IQ2XXS AR · Single user, autoregressive</text>
|
||||
</g>
|
||||
</g>
|
||||
<defs>
|
||||
<clipPath id="p587030b0bd">
|
||||
<rect x="46.899453" y="29.440781" width="223.27" height="162.238516"/>
|
||||
<clipPath id="pe9847d3351">
|
||||
<rect x="46.899453" y="29.440781" width="232.49314" height="162.238516"/>
|
||||
</clipPath>
|
||||
<clipPath id="pc080eae31e">
|
||||
<rect x="331.469609" y="29.440781" width="223.27" height="162.238516"/>
|
||||
<clipPath id="p9e1df01069">
|
||||
<rect x="331.469609" y="29.440781" width="232.49314" height="162.238516"/>
|
||||
</clipPath>
|
||||
</defs>
|
||||
</svg>
|
||||
|
Before Width: | Height: | Size: 28 KiB After Width: | Height: | Size: 31 KiB |
File diff suppressed because it is too large.
Load diff
|
Before Width: | Height: | Size: 39 KiB After Width: | Height: | Size: 43 KiB |
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"schema": "gufo-ds4-boundary-replay/1",
|
||||
"measured_on": "2026-09-24",
|
||||
"test": "ds4_quality_test --indexer-boundary",
|
||||
"build": "Nix gpu-test RelWithDebInfo, assertions enabled",
|
||||
"prompt": {
|
||||
"fixture": "tests/models/deepseek_v4_flash/fixtures/indexer-boundary-prompt.txt",
|
||||
"sha256": "85507b80bee13a6b6a66661b1c4282e30cf597da1dac519841d978b61ff1bef5",
|
||||
"tokens": 2035,
|
||||
"prefill_calls": [
|
||||
2032,
|
||||
3
|
||||
]
|
||||
},
|
||||
"reference": "Gufo autoregressive, identical target weights and fixed token history",
|
||||
"cases": {
|
||||
"dspark_c1": {
|
||||
"generated_tokens": 128,
|
||||
"greedy_mismatches": 0,
|
||||
"full_logit_max_absolute_error": 0
|
||||
},
|
||||
"ar_c2": {
|
||||
"generated_tokens_per_session": 128,
|
||||
"greedy_mismatches": 0,
|
||||
"full_logit_max_absolute_error": 0,
|
||||
"independent_session_logits_identical": true
|
||||
}
|
||||
},
|
||||
"scope": "First ratio-4 transition beyond 512 compressed entries. Does not establish official target-model parity.",
|
||||
"production_http": {
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"prompt_tokens": 2035,
|
||||
"generated_tokens": 128,
|
||||
"ar_dspark_completion_sha256": "5c68d0345d10a072cb0e25d37689205e84b07d4a5fe3d10927b327ac9ba0890a",
|
||||
"prepared_cohorts": {
|
||||
"depth": 0,
|
||||
"prompt_recipe": "matched pp2048 single-user prompt",
|
||||
"workloads": [
|
||||
"mixed",
|
||||
"repetitive"
|
||||
],
|
||||
"concurrency": [
|
||||
1,
|
||||
2,
|
||||
4,
|
||||
6,
|
||||
8
|
||||
],
|
||||
"all_responses_match_fresh_ar": true,
|
||||
"completion_tokens_per_request": 128
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": "dspark",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"file cache reset before each launch with the supplied --drop-caches command",
|
||||
"readiness = HTTP 200 on /ready"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"default": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 35099 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 262144 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"ready_s": 16.408,
|
||||
"ready_s_sd": null,
|
||||
"samples": 1
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"gufo",
|
||||
"serve",
|
||||
"--port",
|
||||
"35099",
|
||||
"--sessions",
|
||||
"1",
|
||||
"llm",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--context",
|
||||
"262144",
|
||||
"--served-model-name",
|
||||
"bench",
|
||||
"--think",
|
||||
"off",
|
||||
"--max-pending-per-client",
|
||||
"8",
|
||||
"--dspark-model",
|
||||
"DeepSeek-V4-Flash-DSpark-support-0731.gguf"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": false,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "loading",
|
||||
"target": "gufo"
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": "dspark",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"antirez/ds4: ds4 0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"file cache reset before each launch with the supplied --drop-caches command",
|
||||
"readiness = HTTP 200 on /v1/models"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"default": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 262144 --port 32989 --host 127.0.0.1 --rocm --dspark --mtp-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"ready_s": 150.874,
|
||||
"ready_s_sd": null,
|
||||
"samples": 1
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"ds4-server",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--ctx",
|
||||
"262144",
|
||||
"--port",
|
||||
"32989",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--rocm",
|
||||
"--dspark",
|
||||
"--mtp-model",
|
||||
"DeepSeek-V4-Flash-DSpark-support-0731.gguf"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "loading",
|
||||
"target": "reference"
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": null,
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"peak device-global hipMemGetInfo used bytes (total - free), sampled every 250 ms during the request; idle_gib is the same counter before the server started",
|
||||
"server mode: ar"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"16K prefix, pp4096 + tg128": {
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 55771 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 262144 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"gib": 93.88,
|
||||
"idle_gib": 2.38
|
||||
},
|
||||
"pp2048 + tg128": {
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 55771 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 262144 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"gib": 93.09,
|
||||
"idle_gib": 2.38
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"gufo",
|
||||
"serve",
|
||||
"--port",
|
||||
"55771",
|
||||
"--sessions",
|
||||
"1",
|
||||
"llm",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--context",
|
||||
"262144",
|
||||
"--served-model-name",
|
||||
"bench",
|
||||
"--think",
|
||||
"off",
|
||||
"--max-pending-per-client",
|
||||
"8"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "memory",
|
||||
"target": "gufo"
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": null,
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"antirez/ds4: ds4 0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"peak device-global hipMemGetInfo used bytes (total - free), sampled every 250 ms during the request; idle_gib is the same counter before the server started",
|
||||
"server mode: ar"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"16K prefix, pp4096 + tg128": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 262144 --port 36697 --host 127.0.0.1 --rocm",
|
||||
"gib": 100.01,
|
||||
"idle_gib": 2.38
|
||||
},
|
||||
"pp2048 + tg128": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 262144 --port 36697 --host 127.0.0.1 --rocm",
|
||||
"gib": 99.96,
|
||||
"idle_gib": 2.38
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"ds4-server",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--ctx",
|
||||
"262144",
|
||||
"--port",
|
||||
"36697",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--rocm"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "memory",
|
||||
"target": "reference"
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
{
|
||||
"schema": "gufo-benchmark-identities/1",
|
||||
"gufo": {
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd",
|
||||
"build": "nix build, ROCm 7.2.3, gfx1151",
|
||||
"production_source_changes_during_measurement": true,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"source_changes": "Sparse-indexer boundary guard in rocm_graph.cpp; full-logit replay recorded in indexer-boundary.json.",
|
||||
"earlier_build": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"retained_rows": "Single AR, loading, AR C1/C2, and mixed DSpark depths >=4096. The boundary guard is not entered by these operations."
|
||||
},
|
||||
"changed_production_files": {
|
||||
"src/models/deepseek_v4_flash/runtime/rocm_graph.cpp": "9699201d4f96ece2b31891be23f514a53cafa8de036b9c2d15c649043ae7c120"
|
||||
}
|
||||
},
|
||||
"reference": {
|
||||
"project": "antirez/ds4",
|
||||
"revision": "0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"server_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"build": "nix build .#ds4-reference, ROCm 7.2.3, gfx1151",
|
||||
"inference_patches": false
|
||||
},
|
||||
"models": {
|
||||
"repository": "antirez/deepseek-v4-gguf",
|
||||
"target": {
|
||||
"file": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"revision": "1cd7b564460821938add0475a60b942c409295e0",
|
||||
"sha256": "ca22ae2f838e14077c22bc1c1417b71b45b5e5a3687bd96c2ac6e17fdb6261c0"
|
||||
},
|
||||
"dspark": {
|
||||
"file": "DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"revision": "e7f04037032990db0346398d249baf9fb9df1ccc",
|
||||
"sha256": "7e319924541db3f7a163ed7e11d7532a70d48228ab59d36cb81e1d4511885360"
|
||||
},
|
||||
"digest_source": "Previously verified identical cached revisions; quality-qualification.json."
|
||||
},
|
||||
"loading": {
|
||||
"mode": "C1 DSpark, context capacity 262144",
|
||||
"cold_files": "POSIX_FADV_DONTNEED on target and sidecar; mincore verified zero resident pages before each launch.",
|
||||
"runtime_libraries": "Not evicted."
|
||||
},
|
||||
"measurement_window_utc": [
|
||||
"2026-09-23",
|
||||
"2026-09-24"
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"artifactType": "servingBenchmark",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash"
|
||||
},
|
||||
"notes": [
|
||||
"Fresh AR d0 completion from single-ar-gufo.json; no duplicate AR speed sweep."
|
||||
],
|
||||
"results": {
|
||||
"c1": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"cases": {
|
||||
"synthetic_prose_pp2048": {
|
||||
"completionHashes": [
|
||||
"5c68d0345d10a072cb0e25d37689205e84b07d4a5fe3d10927b327ac9ba0890a"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"workload": {
|
||||
"id": "single-ar-d0-mixed-control"
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,346 @@
|
||||
{
|
||||
"artifactType": "servingBenchmark",
|
||||
"benchmarkSchema": "gufo.serving-benchmark.v1",
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"metricDefinitions": {
|
||||
"aggregate_output_tokens_per_second": "overall: completion tokens divided by the sum of measured round spans (end-to-end, includes prefill and queueing)",
|
||||
"aggregate_total_tokens_per_second": "overall: actual prefill plus completion tokens divided by the sum of measured round spans; other statistics describe individual rounds",
|
||||
"completion_sha256": "SHA-256 of generated text for cross-concurrency exactness checks; generated text itself is omitted",
|
||||
"decode_tokens_per_second": "completion tokens divided by server decode time",
|
||||
"draft_acceptance": "accepted support-model tokens divided by drafted support-model tokens",
|
||||
"matchedRoundOutputTokensPerSecond": "aggregate output tokens per second over rounds whose completions all matched the reference hashes",
|
||||
"prefill_tokens_per_second": "actual server prefill tokens divided by server prefill time"
|
||||
},
|
||||
"model": {
|
||||
"id": "bench"
|
||||
},
|
||||
"modelBench": {
|
||||
"measuredOn": "2026-09-23",
|
||||
"mode": "ar",
|
||||
"table": "multi-repetition",
|
||||
"target": "gufo"
|
||||
},
|
||||
"notes": [
|
||||
"gufo serve --port 60857 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 4096 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"fresh server per concurrency level"
|
||||
],
|
||||
"reference": {
|
||||
"source": "self-c1"
|
||||
},
|
||||
"results": {
|
||||
"c1": {
|
||||
"aggregate": {
|
||||
"output_tokens_per_second": {
|
||||
"maximum": 10.621775695838998,
|
||||
"median": 10.621775695838998,
|
||||
"minimum": 10.621775695838998,
|
||||
"overall": 10.621775695838998,
|
||||
"p95": 10.621775695838998,
|
||||
"p99": 10.621775695838998
|
||||
},
|
||||
"prefill_tokens_per_second": {
|
||||
"maximum": 168.70367179406784,
|
||||
"median": 168.70367179406784,
|
||||
"minimum": 168.70367179406784,
|
||||
"overall": 168.70367179406784,
|
||||
"p95": 168.70367179406784,
|
||||
"p99": 168.70367179406784
|
||||
},
|
||||
"total_tokens_per_second": {
|
||||
"maximum": 179.32544748990685,
|
||||
"median": 179.32544748990685,
|
||||
"minimum": 179.32544748990685,
|
||||
"overall": 179.32544748990685,
|
||||
"p95": 179.32544748990685,
|
||||
"p99": 179.32544748990685
|
||||
}
|
||||
},
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"cases": {
|
||||
"synthetic_repetition_pp2048": {
|
||||
"acceptance": null,
|
||||
"acceptedPerOutputToken": 0.0,
|
||||
"acceptedTokens": 0,
|
||||
"cacheHits": 0,
|
||||
"completionHashCounts": {
|
||||
"5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860": 1
|
||||
},
|
||||
"completionHashes": [
|
||||
"5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860"
|
||||
],
|
||||
"decode_tokens_per_second": {
|
||||
"maximum": 16.29067156106998,
|
||||
"median": 16.29067156106998,
|
||||
"minimum": 16.29067156106998,
|
||||
"p95": 16.29067156106998,
|
||||
"p99": 16.29067156106998
|
||||
},
|
||||
"draftedPerOutputToken": 0.0,
|
||||
"draftedTokens": 0,
|
||||
"matchesC1": true,
|
||||
"matchesReference": true,
|
||||
"requestAcceptance": {
|
||||
"maximum": null,
|
||||
"median": null,
|
||||
"minimum": null,
|
||||
"p95": null,
|
||||
"p99": null
|
||||
},
|
||||
"requestsWithDrafts": 0
|
||||
}
|
||||
},
|
||||
"categories": {
|
||||
"repetition": {
|
||||
"acceptance": null,
|
||||
"acceptedPerOutputToken": 0.0,
|
||||
"acceptedTokens": 0,
|
||||
"cacheHits": 0,
|
||||
"completionHashCounts": {
|
||||
"5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860": 1
|
||||
},
|
||||
"completionHashes": [
|
||||
"5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860"
|
||||
],
|
||||
"decode_tokens_per_second": {
|
||||
"maximum": 16.29067156106998,
|
||||
"median": 16.29067156106998,
|
||||
"minimum": 16.29067156106998,
|
||||
"p95": 16.29067156106998,
|
||||
"p99": 16.29067156106998
|
||||
},
|
||||
"draftedPerOutputToken": 0.0,
|
||||
"draftedTokens": 0,
|
||||
"requestAcceptance": {
|
||||
"maximum": null,
|
||||
"median": null,
|
||||
"minimum": null,
|
||||
"p95": null,
|
||||
"p99": null
|
||||
},
|
||||
"requestsWithDrafts": 0
|
||||
}
|
||||
},
|
||||
"completionExactness": {
|
||||
"comparedCases": 1,
|
||||
"comparedRequests": 1,
|
||||
"exactRate": 1.0,
|
||||
"exactRequests": 1,
|
||||
"matchedCases": 1,
|
||||
"matchedRoundOutputTokensPerSecond": 10.621775695838998,
|
||||
"matchedRounds": 1
|
||||
},
|
||||
"corpusPromptCount": 1,
|
||||
"executionPlans": [
|
||||
"serial-c1"
|
||||
],
|
||||
"latency": {
|
||||
"client_event_inter_token_ms": {
|
||||
"maximum": 61.866359881906384,
|
||||
"median": 61.866359881906384,
|
||||
"minimum": 61.866359881906384,
|
||||
"p95": 61.866359881906384,
|
||||
"p99": 61.866359881906384
|
||||
},
|
||||
"client_ttft_ms": {
|
||||
"maximum": 4192.395419988316,
|
||||
"median": 4192.395419988316,
|
||||
"minimum": 4192.395419988316,
|
||||
"p95": 4192.395419988316,
|
||||
"p99": 4192.395419988316
|
||||
},
|
||||
"queue_ms": {
|
||||
"maximum": 1.897341,
|
||||
"median": 1.897341,
|
||||
"minimum": 1.897341,
|
||||
"p95": 1.897341,
|
||||
"p99": 1.897341
|
||||
},
|
||||
"request_ms": {
|
||||
"maximum": 12050.715780991595,
|
||||
"median": 12050.715780991595,
|
||||
"minimum": 12050.715780991595,
|
||||
"p95": 12050.715780991595,
|
||||
"p99": 12050.715780991595
|
||||
},
|
||||
"server_inter_token_ms": {
|
||||
"maximum": 61.86847392913384,
|
||||
"median": 61.86847392913384,
|
||||
"minimum": 61.86847392913384,
|
||||
"p95": 61.86847392913384,
|
||||
"p99": 61.86847392913384
|
||||
},
|
||||
"server_max_inter_token_ms": {
|
||||
"maximum": 62.975807,
|
||||
"median": 62.975807,
|
||||
"minimum": 62.975807,
|
||||
"p95": 62.975807,
|
||||
"p99": 62.975807
|
||||
},
|
||||
"server_ttft_ms": {
|
||||
"maximum": 4191.694275,
|
||||
"median": 4191.694275,
|
||||
"minimum": 4191.694275,
|
||||
"p95": 4191.694275,
|
||||
"p99": 4191.694275
|
||||
}
|
||||
},
|
||||
"logicalConcurrency": [
|
||||
1
|
||||
],
|
||||
"measuredSpanMs": 12050.715780991595,
|
||||
"metricsSources": [
|
||||
"gufo"
|
||||
],
|
||||
"physicalExecutionWidths": [
|
||||
1
|
||||
],
|
||||
"requestCount": 1,
|
||||
"roundCount": 1,
|
||||
"rounds": [
|
||||
{
|
||||
"output_tokens_per_second": 10.621775695838998,
|
||||
"prefill_tokens_per_second": 168.70367179406784,
|
||||
"repetition": 0,
|
||||
"sampleCount": 1,
|
||||
"span_ms": 12050.715780991595,
|
||||
"total_tokens_per_second": 179.32544748990685
|
||||
}
|
||||
],
|
||||
"samples": [
|
||||
{
|
||||
"cache_hit": false,
|
||||
"cached_prompt_tokens": 0,
|
||||
"case_id": "synthetic_repetition_pp2048",
|
||||
"category": "repetition",
|
||||
"client_event_inter_token_ms": 61.866359881906384,
|
||||
"client_ttft_ms": 4192.395419988316,
|
||||
"completion_sha256": "5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860",
|
||||
"completion_tokens": 128,
|
||||
"concurrency": 1,
|
||||
"decode_ms": 7857.257419999994,
|
||||
"decode_tokens_per_second": 16.29067156106998,
|
||||
"draft_acceptance": null,
|
||||
"draft_accepted_tokens": 0,
|
||||
"draft_tokens": 0,
|
||||
"execution_plan": "serial-c1",
|
||||
"metrics_source": "gufo",
|
||||
"physical_execution_width": 1,
|
||||
"prefill_ms": 4173.768607999999,
|
||||
"prefill_tokens": 2033,
|
||||
"prefill_tokens_per_second": 487.0897720835032,
|
||||
"prompt_tokens": 2033,
|
||||
"queue_ms": 1.897341,
|
||||
"repetition": 0,
|
||||
"request_index": 0,
|
||||
"requested_logical_concurrency": 1,
|
||||
"server_inter_token_ms": 61.86847392913384,
|
||||
"server_max_inter_token_ms": 62.975807,
|
||||
"server_ttft_ms": 4191.694275,
|
||||
"wall_ms": 12050.715780991595,
|
||||
"whole_request_tokens_per_second": 179.32544748990685
|
||||
}
|
||||
],
|
||||
"scheduledPromptCount": 1,
|
||||
"speculative": {
|
||||
"acceptance": null,
|
||||
"acceptedPerOutputToken": 0.0,
|
||||
"acceptedTokens": 0,
|
||||
"cacheHits": 0,
|
||||
"draftedPerOutputToken": 0.0,
|
||||
"draftedTokens": 0,
|
||||
"requestAcceptance": {
|
||||
"maximum": null,
|
||||
"median": null,
|
||||
"minimum": null,
|
||||
"p95": null,
|
||||
"p99": null
|
||||
},
|
||||
"requestsWithDrafts": 0
|
||||
},
|
||||
"stage": {
|
||||
"decode_tokens_per_second": {
|
||||
"maximum": 16.29067156106998,
|
||||
"median": 16.29067156106998,
|
||||
"minimum": 16.29067156106998,
|
||||
"p95": 16.29067156106998,
|
||||
"p99": 16.29067156106998
|
||||
},
|
||||
"prefill_tokens_per_second": {
|
||||
"maximum": 487.0897720835032,
|
||||
"median": 487.0897720835032,
|
||||
"minimum": 487.0897720835032,
|
||||
"p95": 487.0897720835032,
|
||||
"p99": 487.0897720835032
|
||||
},
|
||||
"whole_request_tokens_per_second": {
|
||||
"maximum": 179.32544748990685,
|
||||
"median": 179.32544748990685,
|
||||
"minimum": 179.32544748990685,
|
||||
"p95": 179.32544748990685,
|
||||
"p99": 179.32544748990685
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"schemaVersion": "1.0.0",
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"warnings": [],
|
||||
"workload": {
|
||||
"cachePrompt": false,
|
||||
"caseIds": [
|
||||
"synthetic_repetition_pp2048"
|
||||
],
|
||||
"categories": [
|
||||
"repetition"
|
||||
],
|
||||
"concurrency": [
|
||||
1
|
||||
],
|
||||
"corpusLayout": "homogeneous",
|
||||
"endpointProfile": "gufo",
|
||||
"id": "deepseek-v4-flash-multi-repetition-gufo-ar",
|
||||
"maxOutputTokens": 128,
|
||||
"mode": "corpus",
|
||||
"paddingPolicy": "repeat-each-case",
|
||||
"pinnedReferenceSlots": false,
|
||||
"prefillFirst": false,
|
||||
"promptCount": 1,
|
||||
"promptGenerator": {
|
||||
"promptSha256": "c2a93a70579121bb64e3083901c6f2426113574a0eefaddb27c5c9bf00f8336d",
|
||||
"recipe": "single-user-d0",
|
||||
"requestedTokens": 2048,
|
||||
"task": "repetition",
|
||||
"templateOverhead": 4,
|
||||
"tokensPerWord": 1.143
|
||||
},
|
||||
"repetitions": 1,
|
||||
"suiteSha256": "2e234a2fe33f522a0042616cac82c268e6a6aa3b3326e063630e8fb21124e354",
|
||||
"temperature": 0.0,
|
||||
"transport": "openai-chat-completions-sse",
|
||||
"warmupMaxOutputTokens": 16,
|
||||
"warmupPolicy": "first-cohort",
|
||||
"warmupRounds": 1
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"schema": "gufo-ds4-reference-qualification/1",
|
||||
"measured_on": "2026-09-24",
|
||||
"reference_revision": "0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"control": {
|
||||
"prompt_recipe": "single-user-d0 prose",
|
||||
"prompt_sha256": "85507b80bee13a6b6a66661b1c4282e30cf597da1dac519841d978b61ff1bef5",
|
||||
"prompt_tokens": 2035,
|
||||
"completion_tokens": 128,
|
||||
"temperature": 0,
|
||||
"thinking": false
|
||||
},
|
||||
"completion_sha256": {
|
||||
"ar_fresh": "a3feec2968666d13af0ac8ee77c7a0d776437a049a5bd4e5ab029e223c93a366",
|
||||
"ar_prepared": "a3feec2968666d13af0ac8ee77c7a0d776437a049a5bd4e5ab029e223c93a366",
|
||||
"dspark_fresh": "b3bbd4a04944d1cb82f9d04d0040425f2b0b9aaa0be6a5b4b427bde3dd8bbd0f",
|
||||
"dspark_prepared": "b3bbd4a04944d1cb82f9d04d0040425f2b0b9aaa0be6a5b4b427bde3dd8bbd0f"
|
||||
},
|
||||
"qualification": {
|
||||
"ar_prepared_replay": true,
|
||||
"dspark_ar_equivalent": false,
|
||||
"dspark_comparison_eligible": false
|
||||
},
|
||||
"limitations": [
|
||||
"The control compares the pinned reference with itself; it does not establish official target-model accuracy.",
|
||||
"ROCm native batching disables DSpark at every configured width; C>1 DSpark is unsupported."
|
||||
],
|
||||
"ar_prepared_concurrency": {
|
||||
"levels": [
|
||||
1,
|
||||
2,
|
||||
4,
|
||||
6,
|
||||
8
|
||||
],
|
||||
"all_responses_match_own_c1": true,
|
||||
"new_prompt_tokens_per_request": 0,
|
||||
"completion_tokens_per_request": 128
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
{
|
||||
"schema": "gufo-reference-smoke/1",
|
||||
"date_utc": "2026-09-23",
|
||||
"purpose": "Bounded build/functionality checks, not throughput or original-model qualification.",
|
||||
"reference": {
|
||||
"project": "antirez/ds4",
|
||||
"revision": "0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"source_hash": "sha256-Bo/td1HwVjw6bwz3BDwTP+ZSudkVFCo2aIVK4axvmXg=",
|
||||
"rocm": "7.2.3",
|
||||
"gpu": "gfx1151",
|
||||
"bench_sha256": "716dab03ddc11169ef55c11632cd5d4cd9e9cdaa546d7798ea0cb9729ac90644",
|
||||
"inference_patches": false
|
||||
},
|
||||
"models": {
|
||||
"target": "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"target_revision": "1cd7b564460821938add0475a60b942c409295e0",
|
||||
"dspark": "DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"dspark_revision": "e7f04037032990db0346398d249baf9fb9df1ccc"
|
||||
},
|
||||
"http": {
|
||||
"context_per_session": 4096,
|
||||
"temperature": 0,
|
||||
"thinking": false,
|
||||
"c1_ar_requests": 2,
|
||||
"c2_ar_requests": 2,
|
||||
"generated_tokens_each": 32,
|
||||
"all_completions_equal": true,
|
||||
"completion_sha256": "5bc351611354785ab17898ed2a703901ff3709459159fb6d8cbf87f56812f79e",
|
||||
"c2_sidecar_loaded_but_speculation_disabled_by_upstream": true,
|
||||
"pp_tg_timings_available": false
|
||||
},
|
||||
"native_bench": {
|
||||
"mode": "dspark",
|
||||
"context_allocation": 4096,
|
||||
"frontiers": [
|
||||
16,
|
||||
32
|
||||
],
|
||||
"prefill_tokens_each": 16,
|
||||
"generated_tokens_each": 8,
|
||||
"snapshot_restore_between_frontiers": true,
|
||||
"finite_positive_csv_rates": true,
|
||||
"supports_multiple_sessions": false
|
||||
},
|
||||
"optional_packages": {
|
||||
"default_gufo_reference_dependency": false,
|
||||
"default_dev_shell_reference_dependency": false,
|
||||
"hosted_pr_check_reference_dependency": false,
|
||||
"llama_release_runtime_unchanged": true,
|
||||
"llama_mtp_runtime_unchanged": true
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-23",
|
||||
"measurement_window_utc": [
|
||||
"2026-09-23",
|
||||
"2026-09-24"
|
||||
],
|
||||
"mode": null,
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"pp2048/tg128; depth is a cached conversation prefix of synthetic text (8-token reply); measured turn task is recorded per row; tolerance max(32, 0.500%) on cache_n and prompt_n; actual counts per sample in rows",
|
||||
"synthetic text: 1.143 tokens/word, template overhead 4 tokens"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"0": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 0,
|
||||
"completion_sha256": "5c68d0345d10a072cb0e25d37689205e84b07d4a5fe3d10927b327ac9ba0890a",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2035
|
||||
}
|
||||
],
|
||||
"pp": 484.62,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 16.28,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"12288": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 12241,
|
||||
"completion_sha256": "a12e2e45d43a23d3cdbddb964708269cf18955c393fdc47c208857b7d34487d3",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2042
|
||||
}
|
||||
],
|
||||
"pp": 393.95,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.64,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"131072": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 131248,
|
||||
"completion_sha256": "c1dda723760ecbb43d6c2b21c86a90e98a9b1d962b9669397eac48426b0b917c",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2052
|
||||
}
|
||||
],
|
||||
"pp": 279.95,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 12.04,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"16384": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 16364,
|
||||
"completion_sha256": "d2afc8c2e9ebf87c60f581843802ef4d6c6aca53bab863c670430c2f661da1d1",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2054
|
||||
}
|
||||
],
|
||||
"pp": 358.41,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.5,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"32768": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 32798,
|
||||
"completion_sha256": "8221920d34d610692373895963d9021228dcdffbbfdc6619315c5c928e311b30",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2035
|
||||
}
|
||||
],
|
||||
"pp": 370.1,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 14.87,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"4096": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 4116,
|
||||
"completion_sha256": "ceb9a340750d2e09c7fb1b1b11533983378235dd1a9d9371a63b57c1cb9e6768",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2040
|
||||
}
|
||||
],
|
||||
"pp": 399.75,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.97,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"65536": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 52501 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 65293,
|
||||
"completion_sha256": "f79f9bb227088507106ffc656c8d6b3c077950d0784e609e51a98d643151b72f",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2047
|
||||
}
|
||||
],
|
||||
"pp": 321.28,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 13.95,
|
||||
"tg_sd": null,
|
||||
"workload": "story"
|
||||
},
|
||||
"8192": {
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 50603 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 8177,
|
||||
"completion_sha256": "ef4ee36cd48f152bf267f3f5147d54732e7b98c9aa144d01f2b6d27975b5e283",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2028
|
||||
}
|
||||
],
|
||||
"pp": 368.17,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.82,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"gufo",
|
||||
"serve",
|
||||
"--port",
|
||||
"52501",
|
||||
"--sessions",
|
||||
"1",
|
||||
"llm",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--context",
|
||||
"133760",
|
||||
"--served-model-name",
|
||||
"bench",
|
||||
"--think",
|
||||
"off",
|
||||
"--max-pending-per-client",
|
||||
"8"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "single-ar",
|
||||
"target": "gufo"
|
||||
}
|
||||
@@ -0,0 +1,221 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": null,
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"antirez/ds4: ds4 0aaea5a238fb41a35106a551e73c8409dfb751ac",
|
||||
"pp2048/tg128; depth is a cached conversation prefix of synthetic text (8-token reply); measured turn task is recorded per row; tolerance max(32, 0.500%) on cache_n and prompt_n; actual counts per sample in rows",
|
||||
"synthetic text: 1.138 tokens/word, template overhead 4 tokens"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"0": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 43089 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 0,
|
||||
"completion_sha256": "a3feec2968666d13af0ac8ee77c7a0d776437a049a5bd4e5ab029e223c93a366",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2035
|
||||
}
|
||||
],
|
||||
"pp": 248.41,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 14.83,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"12288": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 12242,
|
||||
"completion_sha256": "4abb0fe9f5bb2f02cc44528a173d1b650871059445d562b2d84a080d7cbdd386",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2041
|
||||
}
|
||||
],
|
||||
"pp": 212.41,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 13.78,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"131072": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 131249,
|
||||
"completion_sha256": "3600ae1a59f8b53cc4134fd3d7ffea910a327c814c31d8d379170b8db9d0774d",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2051
|
||||
}
|
||||
],
|
||||
"pp": 140.04,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 10.15,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"16384": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 16365,
|
||||
"completion_sha256": "c6333437ae1ed5749f66875de6509bef9679090c76ce81b8d32feb2ebc6ae3dc",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2053
|
||||
}
|
||||
],
|
||||
"pp": 214.97,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 13.64,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"32768": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 32799,
|
||||
"completion_sha256": "ab2b9b7f8925b0dd03ebd0fd726dce0c20ab24073f92faa99fe091b47a3bab40",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2034
|
||||
}
|
||||
],
|
||||
"pp": 205.7,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 12.96,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"4096": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 43089 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 4117,
|
||||
"completion_sha256": "3079cfcd00e0d7d38bbd82703fe850759d0ee10bccd757711d434a679bb60132",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2039
|
||||
}
|
||||
],
|
||||
"pp": 238.93,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 14.12,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"65536": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 65570,
|
||||
"completion_sha256": "ecea793ebd581c802d945d126aea59ddfe3a3313df4574d1f1b84974bc15a29f",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2055
|
||||
}
|
||||
],
|
||||
"pp": 177.43,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 12.0,
|
||||
"tg_sd": null,
|
||||
"workload": "story"
|
||||
},
|
||||
"8192": {
|
||||
"binary_sha256": "43b178469fa14174aa8424c9a0095830dd3a4b85e03d5cd8a9cf5dc2a166507a",
|
||||
"command": "ds4-server --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --ctx 133760 --port 37507 --host 127.0.0.1 --rocm",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 8178,
|
||||
"completion_sha256": "ed2e1059b1c9057b598abafb105772176d5257a09b729595634d2777e800f39b",
|
||||
"draft_n": 0,
|
||||
"draft_n_accepted": 0,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2027
|
||||
}
|
||||
],
|
||||
"pp": 223.24,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 13.9,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"ds4-server",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--ctx",
|
||||
"133760",
|
||||
"--port",
|
||||
"37507",
|
||||
"--host",
|
||||
"127.0.0.1",
|
||||
"--rocm"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "single-ar",
|
||||
"target": "reference"
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"measurement_window_utc": [
|
||||
"2026-09-23",
|
||||
"2026-09-24"
|
||||
],
|
||||
"mode": "dspark",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"pp2048/tg128; depth is a cached conversation prefix of synthetic text (8-token reply); measured turn task is recorded per row; tolerance max(32, 0.500%) on cache_n and prompt_n; actual counts per sample in rows",
|
||||
"synthetic text: 1.143 tokens/word, template overhead 4 tokens"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"0": {
|
||||
"acceptance": 48.65,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.39,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 41307 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 0,
|
||||
"completion_sha256": "5c68d0345d10a072cb0e25d37689205e84b07d4a5fe3d10927b327ac9ba0890a",
|
||||
"draft_n": 74,
|
||||
"draft_n_accepted": 36,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2035
|
||||
}
|
||||
],
|
||||
"pp": 481.58,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 18.25,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"12288": {
|
||||
"acceptance": 42.86,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.08,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 12242,
|
||||
"completion_sha256": "231a3533d80c5f70757b15a6a3292e2719c72a5366a4b156400dd4ff0e3a691f",
|
||||
"draft_n": 21,
|
||||
"draft_n_accepted": 9,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2041
|
||||
}
|
||||
],
|
||||
"pp": 393.32,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.79,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"131072": {
|
||||
"acceptance": 43.48,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.08,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 131249,
|
||||
"completion_sha256": "16c32d552c3f6f7f1deaad2a2417dd5c5dfae88a5e5048c6c41c993d9cc7d816",
|
||||
"draft_n": 23,
|
||||
"draft_n_accepted": 10,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2051
|
||||
}
|
||||
],
|
||||
"pp": 283.45,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 11.95,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"16384": {
|
||||
"acceptance": 38.1,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.07,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 16365,
|
||||
"completion_sha256": "f81b929a5f2b379822da27b81dbac1cf0596b8110263b50f87fea54f42c127c1",
|
||||
"draft_n": 21,
|
||||
"draft_n_accepted": 8,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2053
|
||||
}
|
||||
],
|
||||
"pp": 358.9,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.52,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"32768": {
|
||||
"acceptance": 42.86,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.08,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 32799,
|
||||
"completion_sha256": "d64951af07e05fa60c0afa5ab3761cca0b0a9e8bc2fc3f3c2bbb4ada7b8466e3",
|
||||
"draft_n": 21,
|
||||
"draft_n_accepted": 9,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2034
|
||||
}
|
||||
],
|
||||
"pp": 362.5,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 14.98,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"4096": {
|
||||
"acceptance": 38.1,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.07,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 4117,
|
||||
"completion_sha256": "178743e643f2ec9c6ec636caeca411a27ac29f726c253f031f979990e2bde764",
|
||||
"draft_n": 21,
|
||||
"draft_n_accepted": 8,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2039
|
||||
}
|
||||
],
|
||||
"pp": 398.06,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 16.01,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
},
|
||||
"65536": {
|
||||
"acceptance": 45.83,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.09,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 65570,
|
||||
"completion_sha256": "c5bf4ca61b6f1e445475a5e5213362dc3405c42c6fc679d16a716eb50761f288",
|
||||
"draft_n": 24,
|
||||
"draft_n_accepted": 11,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2055
|
||||
}
|
||||
],
|
||||
"pp": 333.33,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 14.05,
|
||||
"tg_sd": null,
|
||||
"workload": "story"
|
||||
},
|
||||
"8192": {
|
||||
"acceptance": 47.06,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 0.14,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "9fcb41de6ae9739dcdcd249e4742dc4a667adce63030454ac86ba032d49de672",
|
||||
"command": "gufo serve --port 42989 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 8178,
|
||||
"completion_sha256": "2b48bcc119f0244564b0cd1f5e9a914c2b4a97b67b4aa55ac31ae90b433d1856",
|
||||
"draft_n": 34,
|
||||
"draft_n_accepted": 16,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2027
|
||||
}
|
||||
],
|
||||
"pp": 367.84,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 16.48,
|
||||
"tg_sd": null,
|
||||
"workload": "prose"
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"gufo",
|
||||
"serve",
|
||||
"--port",
|
||||
"41307",
|
||||
"--sessions",
|
||||
"1",
|
||||
"llm",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--context",
|
||||
"133760",
|
||||
"--served-model-name",
|
||||
"bench",
|
||||
"--think",
|
||||
"off",
|
||||
"--max-pending-per-client",
|
||||
"8",
|
||||
"--dspark-model",
|
||||
"DeepSeek-V4-Flash-DSpark-support-0731.gguf"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "single-dspark",
|
||||
"target": "gufo"
|
||||
}
|
||||
@@ -0,0 +1,261 @@
|
||||
{
|
||||
"artifactType": "model-bench-table",
|
||||
"fingerprint": {
|
||||
"canonical": {
|
||||
"cppStandard": "C++20",
|
||||
"cpuArchitecture": "x86_64",
|
||||
"cpuLogicalCores": 32,
|
||||
"cpuModel": "AMD RYZEN AI MAX+ 395 w/ Radeon 8060S",
|
||||
"cpuPhysicalCores": 16,
|
||||
"cxxCompiler": "GCC 15.3.0",
|
||||
"gpuArchitecture": "gfx1151",
|
||||
"gpuComputeUnits": 20,
|
||||
"gpuDriver": "amdgpu",
|
||||
"gpuName": "AMD Radeon 8060S Graphics",
|
||||
"gpuPciId": "1002:1586",
|
||||
"kernelRelease": "7.1.8",
|
||||
"memoryTotalBytes": 134309883904,
|
||||
"memoryType": "LPDDR5X (Unified)",
|
||||
"rocmVersion": "7.2.3",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"fingerprintId": "46337fb568b92fb76472ec90a0af584eb7d7ae401f9878d8d18b09762e7c4caa",
|
||||
"schemaVersion": "1.0.0"
|
||||
},
|
||||
"measuredOn": "2026-09-24",
|
||||
"mode": "dspark",
|
||||
"model": {
|
||||
"id": "DeepSeek-V4-Flash",
|
||||
"variant": null
|
||||
},
|
||||
"notes": [
|
||||
"pp2048/tg128; depth is a cached conversation prefix of synthetic text (8-token reply); measured turn task is recorded per row; tolerance max(32, 0.500%) on cache_n and prompt_n; actual counts per sample in rows",
|
||||
"synthetic text: 1.138 tokens/word, template overhead 4 tokens"
|
||||
],
|
||||
"referenceName": "antirez/ds4",
|
||||
"rows": {
|
||||
"0": {
|
||||
"acceptance": 61.81,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.28,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 43459 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 0,
|
||||
"completion_sha256": "5f24c5392d1f7daf95ba57db154752b6ff7cf0c4d491eb2fecd288a3d6572860",
|
||||
"draft_n": 144,
|
||||
"draft_n_accepted": 89,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2033
|
||||
}
|
||||
],
|
||||
"pp": 483.11,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 26.62,
|
||||
"tg_sd": null,
|
||||
"workload": "repetition"
|
||||
},
|
||||
"12288": {
|
||||
"acceptance": 58.11,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.05,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 12242,
|
||||
"completion_sha256": "c4ab5d3cebeb51a35f9db091508303d939df41a91df36abed2f3649664493240",
|
||||
"draft_n": 148,
|
||||
"draft_n_accepted": 86,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2043
|
||||
}
|
||||
],
|
||||
"pp": 356.99,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 23.87,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"131072": {
|
||||
"acceptance": 59.72,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.05,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 131222,
|
||||
"completion_sha256": "43f06b2717b27d993e7835433145b290484a2cd24376d79d1da86225ff26fbc7",
|
||||
"draft_n": 144,
|
||||
"draft_n_accepted": 86,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2053
|
||||
}
|
||||
],
|
||||
"pp": 278.15,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 15.17,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"16384": {
|
||||
"acceptance": 57.89,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.2,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 16362,
|
||||
"completion_sha256": "70a8f45f44416622393f71231df587bf90204ae9c9c953339c2e354cc6a3ecfa",
|
||||
"draft_n": 152,
|
||||
"draft_n_accepted": 88,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2054
|
||||
}
|
||||
],
|
||||
"pp": 355.68,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 23.81,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"32768": {
|
||||
"acceptance": 61.38,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.28,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 32794,
|
||||
"completion_sha256": "08d2923ae28139962e19b060b8c55968db190ab350b2f888a3e709d5172db00f",
|
||||
"draft_n": 145,
|
||||
"draft_n_accepted": 89,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2034
|
||||
}
|
||||
],
|
||||
"pp": 367.12,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 22.9,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"4096": {
|
||||
"acceptance": 60.27,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.2,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 59861 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 4117,
|
||||
"completion_sha256": "b469de3c7d4e46b23dc2dc0b4825659c984f07e88d3b963b8d4f7efb1fb4ee95",
|
||||
"draft_n": 146,
|
||||
"draft_n_accepted": 88,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2041
|
||||
}
|
||||
],
|
||||
"pp": 396.54,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 25.54,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"65536": {
|
||||
"acceptance": 60.69,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.2,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 65561,
|
||||
"completion_sha256": "c79fc98ad8d3b3ded108027bc1d54abcc575dfb63b4f9fa4c08361012ac103fb",
|
||||
"draft_n": 145,
|
||||
"draft_n_accepted": 88,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2058
|
||||
}
|
||||
],
|
||||
"pp": 336.67,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 19.76,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
},
|
||||
"8192": {
|
||||
"acceptance": 61.27,
|
||||
"acceptance_sd": null,
|
||||
"accepted_per_step": 2.12,
|
||||
"accepted_per_step_sd": null,
|
||||
"binary_sha256": "db424fea727a9939ede90b3075d4d7e63bf420f76bd4ebcb6c312fce10cc2312",
|
||||
"command": "gufo serve --port 50293 --sessions 1 llm --model DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --context 133760 --served-model-name bench --think off --max-pending-per-client 8 --dspark-model DeepSeek-V4-Flash-DSpark-support-0731.gguf",
|
||||
"counts": [
|
||||
{
|
||||
"cache_n": 8178,
|
||||
"completion_sha256": "ffbc03e8788264f9c82f4fd48930010f86dc174c8327c7465d67118e08fcf5ae",
|
||||
"draft_n": 142,
|
||||
"draft_n_accepted": 87,
|
||||
"predicted_n": 128,
|
||||
"prompt_n": 2026
|
||||
}
|
||||
],
|
||||
"pp": 366.37,
|
||||
"pp_sd": null,
|
||||
"samples": 1,
|
||||
"tg": 25.03,
|
||||
"tg_sd": null,
|
||||
"workload": "copy"
|
||||
}
|
||||
},
|
||||
"schemaVersion": 1,
|
||||
"server": {
|
||||
"command": [
|
||||
"gufo",
|
||||
"serve",
|
||||
"--port",
|
||||
"50293",
|
||||
"--sessions",
|
||||
"1",
|
||||
"llm",
|
||||
"--model",
|
||||
"DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf",
|
||||
"--context",
|
||||
"133760",
|
||||
"--served-model-name",
|
||||
"bench",
|
||||
"--think",
|
||||
"off",
|
||||
"--max-pending-per-client",
|
||||
"8",
|
||||
"--dspark-model",
|
||||
"DeepSeek-V4-Flash-DSpark-support-0731.gguf"
|
||||
]
|
||||
},
|
||||
"source": {
|
||||
"buildMode": "nix-release",
|
||||
"dirty": true,
|
||||
"revision": "bb6b59ee2dac055460af8949ec469725406ee4dd"
|
||||
},
|
||||
"table": "single-dspark-repetition",
|
||||
"target": "gufo"
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"multi-mixed": {
|
||||
"2": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"4": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"6": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"8": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"1": "Pinned antirez/ds4 C1 DSpark fails greedy AR-equivalence on the matched pp2048/tg128 control, after both fresh and prepared prefill."
|
||||
},
|
||||
"multi-repetition": {
|
||||
"2": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"4": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"6": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"8": "Pinned antirez/ds4 0aaea5a2 disables DSpark in ROCm native batching.",
|
||||
"1": "Pinned antirez/ds4 C1 DSpark fails greedy AR-equivalence on the matched pp2048/tg128 control, after both fresh and prepared prefill."
|
||||
},
|
||||
"single-dspark": {
|
||||
"*": "Pinned antirez/ds4 C1 DSpark fails greedy AR-equivalence on the matched pp2048/tg128 control, after both fresh and prepared prefill."
|
||||
},
|
||||
"single-dspark-repetition": {
|
||||
"*": "Pinned antirez/ds4 C1 DSpark fails greedy AR-equivalence on the matched pp2048/tg128 control, after both fresh and prepared prefill."
|
||||
}
|
||||
}
|
||||
@@ -25,66 +25,9 @@
|
||||
pkgs.${system}.callPackage ./.devops/nix/scope.nix { inherit version; }
|
||||
);
|
||||
|
||||
# Reference LLM runtime for benchmark comparisons (llama-server,
|
||||
# llama-bench), built for gfx1151 with unified memory like the Strix
|
||||
# Halo toolbox. The source is pinned to a release that carries DFlash2
|
||||
# (#27816), Qwen3.8-Flash-Next (#27742) and DSpark support; nixpkgs'
|
||||
# b10273 predates them.
|
||||
llamaCppVersion = "11069"; # release tag b11069, 2026-09-21
|
||||
llamaCppCommit = "68d9053afd4f4d0752ced6187585f862355a40be";
|
||||
llamaCpp = system:
|
||||
(pkgs.${system}.llama-cpp.override {
|
||||
rocmSupport = true;
|
||||
rocmGpuTargets = [ "gfx1151" ];
|
||||
}).overrideAttrs
|
||||
(oldAttrs: {
|
||||
version = llamaCppVersion;
|
||||
src = pkgs.${system}.fetchFromGitHub {
|
||||
owner = "ggml-org";
|
||||
repo = "llama.cpp";
|
||||
rev = llamaCppCommit;
|
||||
hash = "sha256-BnGWYIkVe9y4aufhS5s3Jco1j/BY0MgBlzXrRQFMW3o=";
|
||||
};
|
||||
npmDepsHash = "sha256-2Q7XhaLAArmviOLdQsNbYTfdyDE5pW9lR26cRHEVl9k=";
|
||||
cmakeFlags = (oldAttrs.cmakeFlags or [ ]) ++ [
|
||||
"-DLLAMA_HIP_UMA=ON" # unified memory
|
||||
# The fetched tree has no git metadata; stamp the release commit
|
||||
# so `llama-server --version` identifies the build in artifacts.
|
||||
"-DLLAMA_BUILD_COMMIT:STRING=${builtins.substring 0 8 llamaCppCommit}"
|
||||
];
|
||||
# Pin the ROCm path explicitly and raise the local unroll
|
||||
# threshold for gfx1151 kernels.
|
||||
preConfigure = (oldAttrs.preConfigure or "") + ''
|
||||
cmakeFlagsArray+=("-DCMAKE_HIP_FLAGS=--rocm-path=${pkgs.${system}.rocmPackages.clr} -mllvm --amdgpu-unroll-threshold-local=600")
|
||||
'';
|
||||
});
|
||||
|
||||
# llama.cpp built from the open Qwen3.8-Flash-Next MTP pull request
|
||||
# (ggml-org/llama.cpp#28243), exposed as `llama-server-mtp` next to the
|
||||
# release build so only the sidecar-dependent cells use it. Drop this
|
||||
# once the release pin includes the merged change.
|
||||
llamaCppMtpCommit = "6fcaa16f4b360649933a54d1f91ad40ed35c0e11";
|
||||
llamaCppMtp = system:
|
||||
(llamaCpp system).overrideAttrs (oldAttrs: {
|
||||
pname = "llama-cpp-mtp";
|
||||
version = "11069"; # LLAMA_BUILD_NUMBER must be numeric; the PR branch is based near b11069
|
||||
src = pkgs.${system}.fetchFromGitHub {
|
||||
owner = "danielhanchen";
|
||||
repo = "llama.cpp";
|
||||
rev = llamaCppMtpCommit;
|
||||
hash = "sha256-YgIkYHiV1LNA1OvTcB8SSdOeqr37+cEFAOQjEBfXcK4=";
|
||||
};
|
||||
npmDepsHash = "sha256-2Q7XhaLAArmviOLdQsNbYTfdyDE5pW9lR26cRHEVl9k=";
|
||||
cmakeFlags = builtins.filter (f: !(pkgs.${system}.lib.hasPrefix "-DLLAMA_BUILD_COMMIT" f)) (oldAttrs.cmakeFlags or [ ]) ++ [
|
||||
"-DLLAMA_BUILD_COMMIT:STRING=${builtins.substring 0 8 llamaCppMtpCommit}"
|
||||
];
|
||||
});
|
||||
llamaCppMtpBin = system:
|
||||
pkgs.${system}.runCommand "llama-cpp-mtp-bin" { } ''
|
||||
mkdir -p $out/bin
|
||||
ln -s ${llamaCppMtp system}/bin/llama-server $out/bin/llama-server-mtp
|
||||
ln -s ${llamaCppMtp system}/bin/llama-bench $out/bin/llama-bench-mtp
|
||||
'';
|
||||
llamaReferences = forAllSystems (
|
||||
system: pkgs.${system}.callPackage ./.devops/nix/llama-cpp-reference.nix { }
|
||||
);
|
||||
|
||||
alexnetWeights = system:
|
||||
pkgs.${system}.fetchurl {
|
||||
@@ -171,6 +114,11 @@
|
||||
in
|
||||
{
|
||||
default = base;
|
||||
# Optional benchmark packages: excluded from Gufo, the default
|
||||
# development shell and hosted checks.
|
||||
ds4-reference = pkgs.${system}.callPackage ./.devops/nix/ds4-reference.nix { };
|
||||
llama-cpp-reference = llamaReferences.${system}.release;
|
||||
llama-cpp-mtp-reference = llamaReferences.${system}.mtp;
|
||||
}
|
||||
);
|
||||
|
||||
@@ -188,9 +136,6 @@
|
||||
pkgs.${system}.sox
|
||||
pkgs.${system}.rocmPackages.rocprofiler-sdk
|
||||
pkgs.${system}.sqlite
|
||||
# Benchmark comparison baselines (see .agents/skills/benchmark-model).
|
||||
(llamaCpp system)
|
||||
(llamaCppMtpBin system)
|
||||
audio-cpp.packages.${system}.rocm-gfx1151
|
||||
];
|
||||
env = {
|
||||
|
||||
@@ -3244,7 +3244,14 @@ static bool rocm_graph_encode_layer_attention_batch(
|
||||
}
|
||||
if (ratio == 4) GUFO_DEEPSEEK_ROCM_PROFILE_ATTN_STAGE("indexer_setup");
|
||||
|
||||
if (ok && !zero_prefix && n_tokens <= g->raw_cap) {
|
||||
// A verifier block may cross the first sparse-indexer boundary. Rows
|
||||
// before it still use scalar's dense compressed attention order.
|
||||
const bool crosses_indexer_boundary =
|
||||
g->spec_capture_prefixes && ratio == 4 &&
|
||||
n_comp > DS4_N_INDEXER_TOP_K &&
|
||||
(pos0 + 1u) / ratio <= DS4_N_INDEXER_TOP_K;
|
||||
if (ok && !zero_prefix && n_tokens <= g->raw_cap &&
|
||||
!crosses_indexer_boundary) {
|
||||
const uint32_t n_raw = rocm_graph_raw_span_for_batch(g, pos0, n_tokens);
|
||||
/* See the raw-only branch above: batched mixed attention also
|
||||
* consumes a logical raw window, linearized out of the ring. */
|
||||
|
||||
@@ -36,6 +36,7 @@ if(ENGINE_ENABLE_HIP)
|
||||
add_executable(ds4_quality_test quality_test.cpp token_goldens.cpp
|
||||
official_reference.cpp)
|
||||
target_compile_definitions(ds4_quality_test PRIVATE
|
||||
GUFO_DS4_BOUNDARY_PROMPT="${CMAKE_CURRENT_SOURCE_DIR}/fixtures/indexer-boundary-prompt.txt"
|
||||
GUFO_DS4_OFFICIAL_FIXTURE="${CMAKE_CURRENT_SOURCE_DIR}/fixtures/official-0731.json"
|
||||
GUFO_CHAT_TEMPLATE_HF_GOLDENS="${PROJECT_SOURCE_DIR}/tests/fixtures/chat_template_hf_goldens.json")
|
||||
target_link_libraries(ds4_quality_test PRIVATE gufo_deepseek_v4_flash)
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
Music bends stories the firelight instead apples. And when boats evening of for the shared wheat and children firelight.
|
||||
|
||||
Bread along hills the town western and family and. And firelight watch once the town who by who of snow. Were and wheat instead harvest closed autumn for traded. For traded when that still who road toward before autumn the each.
|
||||
|
||||
Past quiet clouds bends watch western. Distant the and mend rain for family count every remember and instead. Clouds the valley town and of and and. Each apples older and that distant. Once mills autumn past western fences and a count brings stone.
|
||||
|
||||
Rain coins ground gather remember before boats still family the tables ground children. And road and argue autumn western who wheat and old town carry timber. Festival timber bridge festival the quiet wool gather and traded harvest tables hills weeks. Distant instead weeks fences old festival children. Over river stories valley the closed before timber when evening bends wheat over lanterns. Hills mills climb along apples town mills clouds. Count older by hills rain firelight farmers each. Climb distant weeks tables bridge count town along family and each before. Stories music salt traded mend wool were rain. Wheat snow clouds gather and argue.
|
||||
|
||||
Were and traded climb distant rain where markets of the river.
|
||||
|
||||
The cider clouds lanterns once children western over toward.
|
||||
|
||||
And count when firelight music the about weeks cider. Every before the traded weeks the. Wool stories valley about fences watch coins still a the winters road western.
|
||||
|
||||
Brings apples children over distant long sheep quiet long of. While cider road were music of toward shared who to along were. Snow over winters stone ground the while and. Music mend once markets distant along and and the.
|
||||
|
||||
Cider quiet and coins snow boats ground long mills hills bread.
|
||||
|
||||
Older neighbours tables stone and argue salt of distant remember traded hills mills. For and timber harvest coins long.
|
||||
|
||||
Tables neighbours neighbours count and the.
|
||||
|
||||
Of neighbours valley music boats hills. Harvest coins watch past shared harvest music by past salt the and road. Harvest the toward cheese and for and apples the mills river about. That were rain town by instead mend traded old for. Gather the bread timber lanterns coins and when. Carry the instead remember firelight music while.
|
||||
|
||||
When climb town the along western boats stone traded where sheep argue. Still firelight hills by before climb timber past boats winters traded mills. Evening toward firelight quiet and festival.
|
||||
|
||||
Autumn where traded harvest the over festival long boats snow to climb. That markets shared children hills hills bends brings music carry long argue family. Argue and distant once fences along of western along road were over. The bends and toward while winters climb river toward. Where older still that and fences who long watch and about that. Each still quiet neighbours by the quiet to the hills were. Markets when once closed of count before river past the older farmers brings bends.
|
||||
|
||||
Ground firelight harvest instead distant carry.
|
||||
|
||||
The autumn sheep for markets long and while. While who every children boats family the remember coins. Evening clouds remember firelight mills and older lanterns farmers carry boats over climb the. Wool hills the evening winters to. Fences farmers neighbours remember coins farmers western closed while town and weeks clouds stone.
|
||||
|
||||
Boats argue coins shared to when while western snow and. Shared and road were stories mills toward music children when and cider town cheese. Quiet ground closed cider wheat family apples and for for. Timber cheese children cider apples for for and snow mend boats wheat the weeks. Were weeks who of watch autumn old older lanterns who festival carry once and. Tables distant timber before snow mills. Cheese about over and the ground the long of harvest. Boats bridge and and bread a apples watch that. Firelight the by salt family along remember stone markets over road sheep to. Salt river wheat and by climb argue. Sheep coins town and wheat gather. Apples each the old the each valley bridge of salt coins evening.
|
||||
|
||||
And once bends western bridge lanterns brings autumn stone of winters. Instead and salt rain and festival the music. When every bridge hills town that river ground rain sheep bends the wool road. By and wheat river the markets over. Boats lanterns carry cheese lanterns ground wool were toward bends clouds snow bread town. Wool every road the family old tables and once weeks wool rain every. For rain the while road harvest bread by town. And climb wheat markets gather wheat farmers. Watch and stone and children count cider winters and when. For over the every valley while and. That and cheese distant timber stone gather cheese shared long clouds the old ground. Salt to when for the ground firelight farmers the children tables and. Winters shared who who and before when sheep stories. That markets count traded closed winters.
|
||||
|
||||
Bends over valley distant over long children past. Closed ground sheep lanterns of harvest over count wool by gather. Closed brings festival where apples while mills and. Climb valley hills harvest along farmers cider brings. Rain who family the distant bends mills bread older bridge were. Bread family of the valley gather watch closed the hills by wool still. Festival carry were road a while cheese the climb. Firelight bends along the were the count before harvest. Farmers neighbours evening when each brings older children valley. The quiet the distant carry apples wool markets apples and the river. Stone valley to neighbours and valley timber bends. Sheep each before before were brings to when before snow children family stone. The toward valley lanterns while salt for mills evening for old tables.
|
||||
|
||||
Harvest winters stories coins evening and were and the harvest.
|
||||
|
||||
Firelight that the by every the wheat. Along evening about the hills and ground the tables. Cheese while shared lanterns mend who quiet wheat were harvest of. Watch the a were and snow old neighbours traded and apples timber valley mend. Toward cider western by the valley road evening and evening lanterns before. Coins about argue the clouds past cider for mills once weeks. Gather and past closed carry coins by lanterns stone.
|
||||
|
||||
The climb sheep watch tables a ground lanterns by. Of clouds each apples rain town firelight older watch. Gather the children markets firelight where mend a. Clouds before mend bridge farmers the stone once and. Winters every and climb bridge carry. Stone timber valley the were climb shared quiet shared mills while watch about valley. To coins before brings remember remember timber wool bends the of wheat and.
|
||||
|
||||
And old town climb by valley carry music instead along once long town timber. When while gather and who bends where valley bread music traded timber. Where and where and harvest watch remember autumn about before boats snow traded cheese. Before for markets older instead and to bread traded coins. For and still distant each the the clouds gather.
|
||||
|
||||
River tables of mills festival quiet over. Mills long once neighbours festival instead wheat and mills.
|
||||
|
||||
Of and neighbours bends brings of and markets and sheep. Cider carry evening timber remember a farmers evening tables. For apples older evening the past along stone evening. For along road argue and while firelight ground. Closed wheat brings and farmers road snow. Ground that where boats brings lanterns every and stories ground where bread. Quiet apples hills past timber toward road count family.
|
||||
|
||||
Boats traded wheat harvest watch were fences for apples brings. The brings older coins lanterns and instead stories. By and music when firelight gather along each shared count salt a. Markets cider music climb harvest closed traded long. Music for that for long timber who hills the and town.
|
||||
|
||||
Toward to timber bridge brings and along old. Count bends wool winters markets children were.
|
||||
|
||||
Markets remember and watch bridge were children apples stone.
|
||||
|
||||
Long stone of about firelight hills farmers clouds sheep. Bridge bends brings ground coins about town hills.
|
||||
|
||||
And hills markets town sheep music quiet river clouds a. Road valley to that before bends boats winters count shared salt. Remember the for past the every and by music brings sheep remember brings. Wool and and the mend autumn for apples carry weeks town harvest family. Watch ground bread climb the by the music apples boats by gather and a. Once old toward distant of who once carry of where and once the older.
|
||||
|
||||
Evening about stories neighbours long markets closed and gather children toward and cheese once. The and the ground over timber evening bridge argue gather climb and firelight. And neighbours watch and shared shared toward before distant bridge. Town gather who valley for cheese sheep rain by traded festival valley mend shared. Ground river gather watch cheese weeks when town that. And of old tables bends road while timber and weeks farmers. Timber gather weeks where quiet count. Carry and long about apples coins bridge river toward snow a remember weeks. Brings shared every while by bread bends instead the neighbours quiet tables. Western clouds festival that shared before over timber when evening and mills. Sheep older autumn ground gather older shared bridge where and river where once and. By wool bridge when of remember along and cheese.
|
||||
|
||||
While fences the bends where toward harvest of instead of traded past. River stone past argue of for western once by before. Western fences western brings winters who instead harvest the for. Stories closed boats once still markets. A apples by festival by salt who brings.
|
||||
|
||||
The quiet wheat were ground watch neighbours gather evening over and. Town stories coins wheat children town music gather.
|
||||
|
||||
Ground old quiet carry autumn family by to autumn. Older winters for bends family town.
|
||||
|
||||
Shared by brings timber were cheese cider. Valley the were markets instead were western the western. Boats and bridge stone of clouds distant still and and cider along by. By tables while of and past children festival stone long winters wheat and. Of western and climb music winters bends timber instead.
|
||||
|
||||
Markets the stone and stone by. Festival still neighbours a carry cheese along the toward over of a family bridge. Count climb apples apples firelight the children closed that closed western to about. Along count fences of still carry still the coins. Rain a toward apples by count while argue. Weeks each that a for closed harvest salt road road hills stone carry. Argue gather gather who fences carry hills family quiet shared climb the. Quiet markets once once carry cider of the. When closed hills where to road bends for and apples past.
|
||||
|
||||
Summarize the passage above in detail, then write a short story inspired by it. Write at least 500 words.
|
||||
@@ -5,7 +5,9 @@
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
#include <exception>
|
||||
#include <fstream>
|
||||
#include <iostream>
|
||||
#include <iterator>
|
||||
#include <limits>
|
||||
#include <memory>
|
||||
#include <span>
|
||||
@@ -39,6 +41,92 @@ void Expect(bool condition, const std::string& message) {
|
||||
Expect(condition, message.c_str());
|
||||
}
|
||||
|
||||
void CheckIndexerBoundary(
|
||||
const std::shared_ptr<gufo::models::deepseek_v4_flash::Model>& model,
|
||||
bool batched) {
|
||||
// Shared pp2048 benchmark recipe, seed 100000, at 2035 actual tokens.
|
||||
// Generation crosses ratio-4's 512-entry sparse attention boundary.
|
||||
std::ifstream file(GUFO_DS4_BOUNDARY_PROMPT);
|
||||
Expect(file.good(), "open indexer boundary prompt");
|
||||
const std::string text{std::istreambuf_iterator<char>(file), {}};
|
||||
const auto prompt = model->EncodeChat("", text);
|
||||
Expect(prompt.size() == 2035, "indexer boundary prompt token identity");
|
||||
std::string error;
|
||||
auto ar = model->CreateSession(gufo::core::SessionMode::kAutoregressive,
|
||||
133760, &error);
|
||||
auto spec =
|
||||
model->CreateSession(batched ? gufo::core::SessionMode::kAutoregressive
|
||||
: gufo::core::SessionMode::kSpeculative,
|
||||
133760, &error);
|
||||
auto peer =
|
||||
batched ? model->CreateSession(gufo::core::SessionMode::kAutoregressive,
|
||||
133760, &error)
|
||||
: nullptr;
|
||||
Expect(ar && spec, error);
|
||||
// Match HTTP's stable prompt checkpoint followed by its final three tokens.
|
||||
std::vector<int> prefix(prompt.begin(), prompt.end() - 3);
|
||||
Expect(ar->Sync(prefix, &error) && ar->Sync(prompt, &error), error);
|
||||
Expect(spec->Sync(prefix, &error) && spec->Sync(prompt, &error), error);
|
||||
if (batched)
|
||||
Expect(peer && peer->Sync(prefix, &error) && peer->Sync(prompt, &error),
|
||||
error);
|
||||
float worst_error = 0;
|
||||
const auto compare = [&](std::size_t generated) {
|
||||
const auto a = ar->CopyLogits(&error);
|
||||
const auto b = spec->CopyLogits(&error);
|
||||
if (peer)
|
||||
Expect(peer->CopyLogits(&error) == b, "independent AR batch rows agree");
|
||||
Expect(!a.empty() && a.size() == b.size(), "replay logit shape");
|
||||
double squared = 0;
|
||||
float maximum = 0;
|
||||
for (std::size_t i = 0; i < a.size(); ++i) {
|
||||
Expect(std::isfinite(a[i]) && std::isfinite(b[i]),
|
||||
"finite replay logits");
|
||||
const double delta = static_cast<double>(a[i]) - b[i];
|
||||
squared += delta * delta;
|
||||
maximum = std::max(maximum, std::abs(a[i] - b[i]));
|
||||
}
|
||||
if (maximum != 0 && worst_error == 0)
|
||||
std::cout << "Indexer boundary first logit difference at=" << generated
|
||||
<< " max_error=" << maximum
|
||||
<< " rmse=" << std::sqrt(squared / a.size()) << '\n';
|
||||
worst_error = std::max(worst_error, maximum);
|
||||
return maximum == 0;
|
||||
};
|
||||
bool exact = compare(0);
|
||||
std::size_t generated = 0;
|
||||
std::size_t mismatches = 0;
|
||||
while (generated < 128) {
|
||||
std::vector<int> emitted;
|
||||
if (batched) {
|
||||
const int token = spec->SelectNext(0.0F, nullptr);
|
||||
const std::array<gufo::models::deepseek_v4_flash::SessionBatchItem, 2>
|
||||
items{{{spec.get(), token}, {peer.get(), token}}};
|
||||
Expect(model->EvaluateBatch(items, &error), error);
|
||||
emitted.push_back(token);
|
||||
} else {
|
||||
Expect(spec->DsparkStep(128 - generated, 7, &emitted, &error), error);
|
||||
}
|
||||
Expect(!emitted.empty(), "DSpark replay progress");
|
||||
for (const int token : emitted) {
|
||||
const int expected = ar->SelectNext(0.0F, nullptr);
|
||||
if (token != expected) {
|
||||
++mismatches;
|
||||
std::cout << "Prompt replay mismatch at=" << generated
|
||||
<< " candidate=" << token << " expected=" << expected << '\n';
|
||||
}
|
||||
Expect(ar->Evaluate(token, &error), error);
|
||||
++generated;
|
||||
}
|
||||
exact = compare(generated) && exact;
|
||||
}
|
||||
std::cout << "Indexer boundary " << (batched ? "AR C2" : "DSpark C1")
|
||||
<< ": mismatches=" << mismatches << "/128 max_error=" << worst_error
|
||||
<< '\n';
|
||||
Expect(exact && mismatches == 0,
|
||||
"indexer boundary matches scalar tokens/logits");
|
||||
}
|
||||
|
||||
constexpr std::string_view kTrajectoryPrompt =
|
||||
"Continue this technical explanation with concrete examples and "
|
||||
"tradeoffs: Virtual memory lets an operating system";
|
||||
@@ -1317,8 +1405,10 @@ int main(int argc, char** argv) try {
|
||||
|
||||
const bool dspark_replay =
|
||||
argc == 2 && std::string_view(argv[1]) == "--dspark-replay";
|
||||
const bool dspark =
|
||||
dspark_replay || (argc == 2 && std::string_view(argv[1]) == "--dspark");
|
||||
const bool indexer_boundary =
|
||||
argc == 2 && std::string_view(argv[1]) == "--indexer-boundary";
|
||||
const bool dspark = indexer_boundary || dspark_replay ||
|
||||
(argc == 2 && std::string_view(argv[1]) == "--dspark");
|
||||
const bool wide_prefill =
|
||||
argc == 2 && std::string_view(argv[1]) == "--wide-prefill";
|
||||
const bool template_only =
|
||||
@@ -1331,7 +1421,8 @@ int main(int argc, char** argv) try {
|
||||
if (argc > 1 && !dspark && !wide_prefill && !official && !frontiers &&
|
||||
!template_only) {
|
||||
std::cerr << "usage: ds4_quality_test "
|
||||
"[--template-only|--dspark|--dspark-replay|--wide-prefill|--"
|
||||
"[--template-only|--dspark|--dspark-replay|--indexer-boundary|"
|
||||
"--wide-prefill|--"
|
||||
"official OUT.json|"
|
||||
"--reference-frontiers OUT-DIR [--prefill-only]]\n";
|
||||
return 2;
|
||||
@@ -1371,7 +1462,14 @@ int main(int argc, char** argv) try {
|
||||
CheckDsparkScalarQuality(model);
|
||||
return 0;
|
||||
}
|
||||
if (indexer_boundary) {
|
||||
CheckIndexerBoundary(model, false);
|
||||
CheckIndexerBoundary(model, true);
|
||||
return 0;
|
||||
}
|
||||
if (dspark) {
|
||||
CheckIndexerBoundary(model, false);
|
||||
CheckIndexerBoundary(model, true);
|
||||
CheckDsparkReproducibility(model);
|
||||
CheckDsparkChangingMembership(model);
|
||||
CheckConcurrentPolicySnapshot(model);
|
||||
|
||||
@@ -16,7 +16,7 @@ sys.path.insert(0, str(ROOT / "tools"))
|
||||
from gufo import serving_bench
|
||||
from gufo.model_bench.charts import render_charts
|
||||
from gufo.model_bench.config import BenchConfig, load_config
|
||||
from gufo.model_bench.llm import Session, _measure_depth, run_loading, run_multi, run_single
|
||||
from gufo.model_bench.llm import Session, _measure_depth, run_loading, run_multi, run_single, run_table
|
||||
from gufo.model_bench.render import _serving_rate, layout_for, parse_table, render_table
|
||||
from gufo.model_bench.servers import Server
|
||||
|
||||
@@ -515,13 +515,14 @@ def prepared_response(request, timeout):
|
||||
body = json.loads(request.data)
|
||||
slot = body["id_slot"]
|
||||
check(body["cache_prompt"] is True, "both phases allow exact prefix reuse")
|
||||
if body["max_tokens"] == 1:
|
||||
if body["max_tokens"] in (0, 1):
|
||||
generated = body["max_tokens"]
|
||||
with preparation_lock:
|
||||
prepared_slots.add(slot)
|
||||
return FakeResponse(
|
||||
usage={**OPENAI_USAGE, "completion_tokens": 1},
|
||||
timings={**LLAMA_TIMINGS, "prompt_n": 12, "cache_n": 0, "predicted_n": 1},
|
||||
content=("one",),
|
||||
usage={**OPENAI_USAGE, "completion_tokens": generated},
|
||||
timings={**LLAMA_TIMINGS, "prompt_n": 12, "cache_n": 0, "predicted_n": generated},
|
||||
content=("one",) if generated else (),
|
||||
)
|
||||
with preparation_lock:
|
||||
check(prepared_slots == {0, 1}, "all slots are prepared before any tg request")
|
||||
@@ -546,6 +547,14 @@ check(len(prepared_report["results"]["c2"]["samples"]) == 2,
|
||||
check(len(prepared_report["results"]["c2"]["preparations"][0]["samples"]) == 2,
|
||||
"preparation evidence is retained")
|
||||
check(not prepared_report["warnings"], "intentional prepared cache hits are not warnings")
|
||||
prepared_slots.clear()
|
||||
with patch.object(serving_bench.urllib.request, "urlopen", prepared_response):
|
||||
zero_prepared = serving_bench.run_corpus_benchmark(**prepared_args, preparation_tokens=0)
|
||||
check(all(s["completion_tokens"] == 0
|
||||
for s in zero_prepared["results"]["c2"]["preparations"][0]["samples"]),
|
||||
"zero-token preparation retains the prompt frontier without generating a token")
|
||||
check(zero_prepared["workload"]["warmupMaxOutputTokens"] == 0,
|
||||
"the actual preparation policy is recorded")
|
||||
with patch.object(serving_bench.urllib.request, "urlopen",
|
||||
lambda request, timeout: FakeResponse(
|
||||
usage={**OPENAI_USAGE,
|
||||
@@ -831,6 +840,13 @@ with tempfile.TemporaryDirectory() as directory:
|
||||
check((row["Gufo tg mixed"], row["Gain mixed"], row["Gufo tg repetitive"],
|
||||
row["Gain repetitive"]) == ("10.00", "+25.0%", "30.00", "+50.0%"),
|
||||
"generation and gains stay independent by text type")
|
||||
exclusions = config.artifacts_dir / "unavailable.json"
|
||||
exclusions.write_text(json.dumps({workloads[1].id: {"0": "unqualified comparison"}}))
|
||||
excluded = parse_table(render_table(config, single_table, None))["0"]
|
||||
check(excluded["llama.cpp tg repetitive"] == excluded["Gain repetitive"] == "N/A"
|
||||
and excluded["llama.cpp pp"] == "80.00 ± 1.00",
|
||||
"explicit exclusions override retained rates and cannot supply the shared pp maximum")
|
||||
exclusions.unlink()
|
||||
for target, metric, expected in [
|
||||
("gufo", "Gufo tg repetitive", workloads[1]),
|
||||
("reference", "llama.cpp tg mixed", workloads[0]),
|
||||
@@ -960,4 +976,113 @@ with tempfile.TemporaryDirectory() as directory:
|
||||
ar_reference.unlink()
|
||||
table.spec["modes"] = ["ar"]
|
||||
|
||||
# Optional reference builds use their native flags and reject unsupported metrics
|
||||
# before spending time loading the model.
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
config = load_config(ROOT, "deepseek-v4-flash")
|
||||
config.files = {role: {"default": Path(directory) / role} for role in ("gguf", "dspark")}
|
||||
session = Session(
|
||||
config, "reference", gufo_binary=Path("gufo"), reference_binary="ds4-server",
|
||||
source={"revision": "a" * 40, "dirty": False}, fingerprint={},
|
||||
log_dir=Path(directory), document="", todo_only=False,
|
||||
)
|
||||
command = session.reference_command(config.table("multi-ar"), mode="ar",
|
||||
context=4 * 4096, parallel=4, port=8081)
|
||||
check(command[command.index("--ctx") + 1] == "4096"
|
||||
and command[command.index("--batched-session") + 1] == "4",
|
||||
"ds4 context is per session, not the total llama.cpp allocation")
|
||||
command = session.reference_command(config.table("single-dspark"), mode="dspark",
|
||||
context=4096, parallel=1, port=8081)
|
||||
check("--dspark" in command and "--mtp-model" in command and "--batched-session" not in command
|
||||
and "--alias" not in command and "-np" not in command,
|
||||
"C1 DSpark omits the upstream flag that silently disables speculation")
|
||||
try:
|
||||
session.reference_command(config.table("multi-dspark"), mode="dspark",
|
||||
context=8192, parallel=2, port=8081)
|
||||
except RuntimeError as failure:
|
||||
check("disables DSpark" in str(failure), "unsupported speculation must not become an AR baseline")
|
||||
else:
|
||||
raise AssertionError("the pinned reference cannot batch DSpark")
|
||||
session.active_log = Path(directory) / "server.log"
|
||||
session.active_log.write_text("")
|
||||
with patch("gufo.model_bench.llm.run_request") as request, \
|
||||
patch("ds4.server_metrics.request_metrics"):
|
||||
session.request("http://unused", "prompt", 1)
|
||||
check(request.call_args.kwargs["extra_body"]["thinking"] == {"type": "disabled"},
|
||||
"reference HTTP requests disable DeepSeek's default thinking")
|
||||
session.server = MagicMock()
|
||||
config.data["reference"]["stage_timings"] = False
|
||||
try:
|
||||
run_table(session, config.table("single-ar"))
|
||||
except RuntimeError as failure:
|
||||
check("stage timings" in str(failure), "missing reference metrics fail explicitly")
|
||||
else:
|
||||
raise AssertionError("missing reference stage timing must not become a benchmark")
|
||||
check(not session.server.called, "unsupported timing fails before model loading")
|
||||
|
||||
config.artifacts_override = Path(directory)
|
||||
config.files = {"gguf": {"default": Path(__file__)}}
|
||||
table = config.table("multi-ar")
|
||||
table.spec["concurrency"] = [2]
|
||||
session.target = "gufo"
|
||||
session.server = MagicMock(return_value=MagicMock())
|
||||
session.reference_version = lambda mode: None
|
||||
phases = []
|
||||
session.request = lambda *args, **kwargs: phases.append("C1 checkpoint")
|
||||
report = {"artifactType": "servingBenchmark", "workload": {}, "results": {
|
||||
"c2": {"samples": [{"prompt_tokens": 2048, "completion_tokens": 128}] * 2}}}
|
||||
|
||||
def prepared_cohort(**kwargs):
|
||||
phases.append("prepared cohort")
|
||||
check(kwargs["prefill_first"] and kwargs["preparation_tokens"] == 1,
|
||||
"Gufo DS4 still prepares every session before timing decoding")
|
||||
return report
|
||||
|
||||
with patch("gufo.model_bench.llm.Tokenizer", return_value=MagicMock(overhead=4, ratio=1.143)), \
|
||||
patch("gufo.model_bench.llm.run_corpus_benchmark", side_effect=prepared_cohort), \
|
||||
patch("gufo.model_bench.llm.wait_process_exit"), \
|
||||
patch("gufo.model_bench.llm.save_artifact"), \
|
||||
patch("sys.stdout", new=io.StringIO()):
|
||||
run_multi(session, table)
|
||||
check(phases == ["C1 checkpoint", "prepared cohort"],
|
||||
"DS4 builds the scalar prompt checkpoint before concurrent preparation")
|
||||
check(report["results"]["c2"]["promptPreparation"] ==
|
||||
"C1 checkpoint followed by concurrent preparation",
|
||||
"the artifact records the prefill arithmetic control")
|
||||
|
||||
# An AR-only benchmark must not require the optional MTP build on PATH.
|
||||
config = load_config(ROOT, "qwen3.8-flash-next")
|
||||
config.files = {"gguf": {"default": Path("target.gguf")}}
|
||||
session = Session(
|
||||
config, "reference", gufo_binary=Path("gufo"), reference_binary="llama-server",
|
||||
source={"revision": "a" * 40, "dirty": False}, fingerprint={},
|
||||
log_dir=Path("/tmp"), document="", todo_only=False,
|
||||
)
|
||||
with patch("gufo.model_bench.llm.shutil.which", return_value="/bin/llama-server") as which, \
|
||||
patch.object(session, "reference_version"):
|
||||
session.server(config.table("single-ar"), mode="ar", context=4096, sessions=1, tag="test")
|
||||
which.assert_called_once_with("llama-server")
|
||||
|
||||
# Existing server log timers must match HTTP counts; never infer stage time from wall time.
|
||||
from ds4.server_metrics import request_metrics, cohort_metrics
|
||||
log = (
|
||||
"0924 01:00:00 ds4-server: chat ctx=2..12:10 prompt done 0.020s\n"
|
||||
"0924 01:00:00 ds4-server: chat ctx=12..14:2 gen=2 decoding chunk=200.00 t/s avg=200.00 t/s 0.010s\n"
|
||||
)
|
||||
parsed = request_metrics(log, sample)
|
||||
check(parsed.prefill_ms == 20 and parsed.decode_ms == 10, "native stage timers are milliseconds")
|
||||
check(parsed.prefill_tokens_per_second == 500 and parsed.decode_tokens_per_second == 200,
|
||||
"native pp/tg use HTTP counts and independent timers")
|
||||
for invalid in (log + log, log.replace("2..12:10", "0..12:12"), log.replace("0.010s", "0.000s")):
|
||||
try:
|
||||
request_metrics(invalid, sample)
|
||||
except RuntimeError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("ambiguous, mismatched or invalid native timers must fail")
|
||||
cohort = {"samples": [sample.public(), sample.public()], "rounds": [{"sampleCount": 2}]}
|
||||
cohort_metrics(log + log.replace("0.010s", "0.020s"), cohort, 2)
|
||||
check(_serving_rate({"results": {"c2": cohort}}, 2) == 300,
|
||||
"concurrency sums logged per-request rates without inventing client/log ID mapping")
|
||||
|
||||
print("Serving benchmark harness tests passed.")
|
||||
+1
-1
@@ -8,7 +8,7 @@ benchmark/tuning executables; Python tools run from this source tree.
|
||||
|
||||
| Directory | Purpose |
|
||||
| --- | --- |
|
||||
| `bench/` | Kernel microbenchmarks, hipBLASLt tuning, and direct speculative-corpus checks; `model-bench.py` drives per-model BENCHMARKS.md tables against llama.cpp |
|
||||
| `bench/` | Kernel microbenchmarks, hipBLASLt tuning, and direct speculative-corpus checks; `model-bench.py` drives per-model BENCHMARKS.md tables against each model's reference |
|
||||
| `prof/` | rocprofv3 capture, stage summaries, and ISA inspection |
|
||||
| `serving/` | Concurrent HTTP benchmarks and interrupted-chat/disk replay checks |
|
||||
| `ds4/` | DeepSeek-specific validation and experiments |
|
||||
|
||||
@@ -10,6 +10,7 @@ are in [the DS4 benchmark README](../../docs/models/deepseek-v4-flash/BENCHMARKS
|
||||
| `tools/ds4/check.py reference --model "$MODEL" --upstream "$UPSTREAM" --output /tmp/ds4-reference` | Repeated AR scoring against 105 official continuations; independent full logits and 128 forced tokens at 0/4K/8K/12K/16K, with matched 2K/4K prefill calls |
|
||||
| `tools/ds4/check.py reference --prefill-only --model "$MODEL" --upstream "$UPSTREAM" --output /tmp/ds4-prefill` | Same matched prefill grid and exact repeat check; skip continuation scoring and decode replay |
|
||||
| `tools/ds4/check.py all --model "$MODEL" --dspark-model "$DSPARK"` | All maintained DS4 CTest checks, including changing concurrency, snapshots through 16K, mixed sampling, and prefix/disk reuse |
|
||||
| `build/gpu-test/tests/models/deepseek_v4_flash/ds4_quality_test --indexer-boundary` | Focused DSpark C1 / AR C2 full-logit replay across the first sparse-indexer boundary; set `GUFO_DEEPSEEK_V4_FLASH_MODEL` and `GUFO_DEEPSEEK_V4_FLASH_DSPARK_MODEL` |
|
||||
| `tools/ds4/import-eval.py` | Rebuild pinned fixtures from an upstream checkout; default capability subset, `--suite official` for the 0731 continuations |
|
||||
| `result/bin/gufo eval --questions 75 --greedy --output /tmp/ds4-quality.json` | Pinned capability evaluation through the real HTTP server (add `--base-url`) |
|
||||
| `result/bin/gufo bench -c 1,2,4,6,8 -p 2048 -n 128 -d 0,4096,8192,12288,16384 -r 2 -v` | Release model sweep; per-request output hashes and draft counters (add model paths) |
|
||||
@@ -17,6 +18,7 @@ are in [the DS4 benchmark README](../../docs/models/deepseek-v4-flash/BENCHMARKS
|
||||
| `result/bin/gufo bench -c 1,2,4 -p 2048 -n 128 -d 0,4096,8192,12288,16384 -r 2 -v --temperature 0.6 --top-p 0.95 --seed 7` | Filtered sampled sweep; C1 retains AR seed identity, while C>1 uses exact probabilistic proposals |
|
||||
| `tools/ds4/check.py benchmark --concurrency 1,2,4 --temperature 0.6 --top-p 0.95 --seed 7 --ar-log /tmp/ar-t06.log --dspark-log /tmp/dspark-t06.log --output /tmp/bench-t06.json` | Repeated output/counters, AR identity for point-mass cohorts, acceptance and proposal policy |
|
||||
| `tools/serving/gufo-serving-bench.py` | Shared HTTP concurrency, scheduling, and acceptance measurement |
|
||||
| `tools/bench/model-bench.py --model deepseek-v4-flash` | Retained HTTP tables; `server_metrics.py` reads antirez's existing pp/tg log timers |
|
||||
| `tools/bench/speculative-corpus.py` | Shared AR/speculative text comparison on the fixed corpus |
|
||||
| `tools/prof/prof.py` | Shared rocprofv3 capture, rollup, and A/B diff |
|
||||
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Read the pinned antirez server's existing stage timers; no wall-time estimates."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import replace
|
||||
import math
|
||||
import re
|
||||
|
||||
from gufo.serving_bench import RequestObservation
|
||||
|
||||
PREFILL = re.compile(
|
||||
r"ds4-server: chat ctx=(\d+)\.\.(\d+):(\d+).*? prompt done ([\d.]+)s$")
|
||||
DECODE = re.compile(
|
||||
r"ds4-server: chat ctx=\d+\.\.\d+:\d+ gen=(\d+).*? "
|
||||
r"decoding chunk=[\d.]+ t/s avg=[\d.]+ t/s ([\d.]+)s$")
|
||||
|
||||
|
||||
def decode_durations(log: str, tokens: int) -> list[float]:
|
||||
values = [float(match[2]) * 1000 for line in log.splitlines()
|
||||
if (match := DECODE.search(line)) and int(match[1]) == tokens]
|
||||
if any(not math.isfinite(value) or value <= 0 for value in values):
|
||||
raise RuntimeError("antirez reported an invalid decode duration")
|
||||
return values
|
||||
|
||||
|
||||
def request_metrics(log: str, sample: RequestObservation) -> RequestObservation:
|
||||
prefill = [match for line in log.splitlines() if (match := PREFILL.search(line))]
|
||||
decode = decode_durations(log, sample.completion_tokens)
|
||||
if len(prefill) != 1 or len(decode) != 1:
|
||||
raise RuntimeError("cannot identify exactly one antirez prefill/decode timer")
|
||||
start, end, count = map(int, prefill[0].groups()[:3])
|
||||
if (start, end, count) != (sample.cached_prompt_tokens, sample.prompt_tokens, sample.prefill_tokens):
|
||||
raise RuntimeError("antirez log frontier disagrees with HTTP token/cache counts")
|
||||
pp_ms = float(prefill[0][4]) * 1000
|
||||
if not math.isfinite(pp_ms) or pp_ms < 0 or (sample.prefill_tokens and pp_ms == 0):
|
||||
raise RuntimeError("antirez reported an invalid prefill duration")
|
||||
return replace(
|
||||
sample, prefill_ms=pp_ms, decode_ms=decode[0], metrics_source="antirez-server-log",
|
||||
prefill_tokens_per_second=sample.prefill_tokens * 1000 / pp_ms if pp_ms else None,
|
||||
decode_tokens_per_second=sample.completion_tokens * 1000 / decode[0],
|
||||
)
|
||||
|
||||
|
||||
def cohort_metrics(log: str, result: dict, tokens: int) -> None:
|
||||
"""Keep each request's timer without inventing a log-to-client ID mapping."""
|
||||
durations = decode_durations(log, tokens)
|
||||
if len(durations) != len(result["samples"]):
|
||||
raise RuntimeError("antirez decode timer count does not match the completed cohort")
|
||||
result["loggedDecodeTimers"] = {
|
||||
"source": "antirez-server-log", "tokens_per_request": tokens,
|
||||
"durations_ms": durations,
|
||||
}
|
||||
@@ -52,7 +52,7 @@ def new_artifact(
|
||||
"mode": mode,
|
||||
"referenceName": config.reference_name,
|
||||
"model": {"id": config.data["model"]["id"], "variant": table.variant},
|
||||
"measuredOn": dt.date.today().isoformat(),
|
||||
"measuredOn": dt.datetime.now(dt.timezone.utc).date().isoformat(),
|
||||
"source": source,
|
||||
"fingerprint": fingerprint,
|
||||
"server": {"command": public_command(command)},
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from gufo.serving_bench import load_fingerprint, source_identity
|
||||
@@ -116,11 +115,6 @@ def cmd_run(config: BenchConfig, args: argparse.Namespace) -> int:
|
||||
if not gufo_binary.exists():
|
||||
raise SystemExit(f"Gufo binary not found: {gufo_binary} (run `nix build`)")
|
||||
reference_binary = args.reference_binary or config.data["reference"]["server"]
|
||||
if args.target == "reference":
|
||||
binaries = [reference_binary, config.speculative.get("reference", {}).get("server")]
|
||||
for binary in filter(None, binaries):
|
||||
if shutil.which(binary) is None:
|
||||
raise SystemExit(f"{binary} not on PATH; enter `nix develop`")
|
||||
revision, dirty = source_identity(root)
|
||||
fingerprint = load_fingerprint(None, gufo_binary)
|
||||
document = config.benchmarks_path.read_text(encoding="utf-8") if config.benchmarks_path.exists() else ""
|
||||
|
||||
@@ -10,6 +10,7 @@ import json
|
||||
import struct
|
||||
import zlib
|
||||
import random
|
||||
import shutil
|
||||
import statistics
|
||||
import subprocess
|
||||
import threading
|
||||
@@ -99,6 +100,15 @@ class Session:
|
||||
def cache_prompt(self) -> bool | None:
|
||||
return None if self.target == "gufo" else True
|
||||
|
||||
@property
|
||||
def reference_kind(self) -> str:
|
||||
return self.config.data["reference"].get("kind", "llama.cpp")
|
||||
|
||||
@property
|
||||
def request_model(self) -> str:
|
||||
# The upstream alias disables default DeepSeek thinking for corpus requests too.
|
||||
return "deepseek-chat" if self.target == "reference" and self.reference_kind == "ds4" else MODEL_ALIAS
|
||||
|
||||
# ----- server commands -------------------------------------------------
|
||||
|
||||
def gufo_command(self, table: TableSpec, *, mode: str | None, context: int, sessions: int, port: int) -> list[str]:
|
||||
@@ -126,8 +136,19 @@ class Session:
|
||||
cfg = self.config
|
||||
gguf = cfg.file("gguf", table.variant)
|
||||
args = list(cfg.data["reference"]["args"])
|
||||
command = [self.reference_binary_for(mode), "-m", str(gguf), "-c", str(context), "-np", str(parallel),
|
||||
"--port", str(port), "--host", "127.0.0.1", "--alias", MODEL_ALIAS, *args]
|
||||
if self.reference_kind == "ds4":
|
||||
if parallel > 1 and mode and mode != "ar":
|
||||
raise RuntimeError("the pinned ds4 ROCm server disables DSpark in native session batching")
|
||||
# The caller supplies total reference context; ds4 allocates per session.
|
||||
command = [self.reference_binary_for(mode), "--model", str(gguf),
|
||||
"--ctx", str(context // parallel),
|
||||
"--port", str(port), "--host", "127.0.0.1", *args]
|
||||
# Even --batched-session 1 disables DSpark in this upstream pin.
|
||||
if parallel > 1:
|
||||
command += ["--batched-session", str(parallel)]
|
||||
else:
|
||||
command = [self.reference_binary_for(mode), "-m", str(gguf), "-c", str(context), "-np", str(parallel),
|
||||
"--port", str(port), "--host", "127.0.0.1", "--alias", MODEL_ALIAS, *args]
|
||||
if table.kind == "image-encoder":
|
||||
command += ["--mmproj", str(cfg.file("mmproj", table.variant))]
|
||||
if mode and mode != "ar":
|
||||
@@ -145,7 +166,14 @@ class Session:
|
||||
command = self.gufo_command(table, mode=mode, context=context, sessions=sessions, port=placeholder.port)
|
||||
else:
|
||||
command = self.reference_command(table, mode=mode, context=context, parallel=sessions, port=placeholder.port)
|
||||
if shutil.which(command[0]) is None:
|
||||
raise RuntimeError(
|
||||
f"{command[0]} not on PATH; select its optional Nix reference package "
|
||||
"(see docs/BENCHMARKS.md)"
|
||||
)
|
||||
self.reference_version(mode)
|
||||
placeholder.command = command
|
||||
self.active_log = log
|
||||
return placeholder
|
||||
|
||||
def reference_version(self, mode: str | None = None) -> str | None:
|
||||
@@ -154,6 +182,14 @@ class Session:
|
||||
binary = self.reference_binary_for(mode)
|
||||
cache = self.__dict__.setdefault("_reference_versions", {})
|
||||
if binary not in cache:
|
||||
if self.reference_kind == "ds4":
|
||||
executable = Path(shutil.which(binary) or binary).resolve()
|
||||
revision = (executable.parent.parent / "share" / "ds4-revision").read_text().strip()
|
||||
expected = self.config.data["reference"]["revision"]
|
||||
if revision != expected:
|
||||
raise RuntimeError(f"ds4 reference revision {revision} differs from configured {expected}")
|
||||
cache[binary] = f"ds4 {revision}"
|
||||
return cache[binary]
|
||||
completed = subprocess.run([binary, "--version"], capture_output=True, text=True)
|
||||
lines = (completed.stdout + completed.stderr).splitlines()
|
||||
versions = [line.strip() for line in lines if line.strip().startswith("version:")]
|
||||
@@ -176,20 +212,32 @@ class Session:
|
||||
messages: list[dict[str, Any]] | None = None,
|
||||
cache_prompt: bool | None = "default", # type: ignore[assignment]
|
||||
extra_body: dict[str, Any] | None = None) -> RequestObservation:
|
||||
return run_request(
|
||||
base_url=base_url, model=MODEL_ALIAS, prompt=prompt, max_tokens=max_tokens,
|
||||
if self.target == "reference" and self.reference_kind == "ds4":
|
||||
extra_body = {"thinking": {"type": "disabled"}, **(extra_body or {})}
|
||||
use_log = self.target == "reference" and self.reference_kind == "ds4"
|
||||
offset = self.active_log.stat().st_size if use_log else 0
|
||||
sample = run_request(
|
||||
base_url=base_url, model=self.request_model, prompt=prompt, max_tokens=max_tokens,
|
||||
temperature=float(self.config.data["sampling"]["temperature"]),
|
||||
timeout_seconds=REQUEST_TIMEOUT, client_id=CLIENT_ID, concurrency=1,
|
||||
repetition=1, request_index=index, endpoint_profile=self.profile,
|
||||
cache_prompt=self.cache_prompt if cache_prompt == "default" else cache_prompt, messages=messages,
|
||||
extra_body=extra_body,
|
||||
)
|
||||
if use_log:
|
||||
from ds4.server_metrics import request_metrics
|
||||
with self.active_log.open("rb") as log:
|
||||
log.seek(offset)
|
||||
sample = request_metrics(log.read().decode(), sample)
|
||||
return sample
|
||||
|
||||
def chat_text(self, base_url: str, messages: list[dict[str, str]], max_tokens: int,
|
||||
extra_body: dict[str, Any] | None = None) -> str:
|
||||
"""Non-streaming completion text, used to build a reusable conversation prefix."""
|
||||
payload = {"model": MODEL_ALIAS, "messages": messages, "max_tokens": max_tokens,
|
||||
payload = {"model": self.request_model, "messages": messages, "max_tokens": max_tokens,
|
||||
"temperature": float(self.config.data["sampling"]["temperature"]), "stream": False}
|
||||
if self.target == "reference" and self.reference_kind == "ds4":
|
||||
payload["thinking"] = {"type": "disabled"}
|
||||
if self.cache_prompt is not None:
|
||||
payload["cache_prompt"] = self.cache_prompt
|
||||
if extra_body:
|
||||
@@ -358,12 +406,13 @@ def run_single(session: Session, table: TableSpec, display_table: TableSpec | No
|
||||
tgs: list[float] = []
|
||||
accepts: list[float] = []
|
||||
counts: list[dict[str, Any]] = []
|
||||
task = spec.get("depth_workloads", {}).get(str(depth), spec.get("workload", "prose"))
|
||||
try:
|
||||
observations = [
|
||||
_measure_depth(session, server.base_url, tokenizer, depth=depth,
|
||||
prompt_tokens=prompt_tokens, output_tokens=output_tokens,
|
||||
fraction=fraction, seed=base_seed, repetition=repetition,
|
||||
task=spec.get("workload", "prose"))
|
||||
task=task)
|
||||
for repetition in range(repetitions)
|
||||
]
|
||||
except (RuntimeError, OSError) as failure: # OSError: server died mid-request
|
||||
@@ -400,7 +449,7 @@ def run_single(session: Session, table: TableSpec, display_table: TableSpec | No
|
||||
"draft_n": observation.draft_tokens,
|
||||
"draft_n_accepted": accepted,
|
||||
"completion_sha256": observation.completion_sha256})
|
||||
row: dict[str, Any] = {"counts": counts, "samples": repetitions,
|
||||
row: dict[str, Any] = {"counts": counts, "samples": repetitions, "workload": task,
|
||||
"command": " ".join(public_command(server.command))}
|
||||
for name, values in (("pp", pps), ("tg", tgs), ("acceptance", accepts), ("accepted_per_step", per_step)):
|
||||
if values:
|
||||
@@ -412,7 +461,7 @@ def run_single(session: Session, table: TableSpec, display_table: TableSpec | No
|
||||
wait_process_exit(server)
|
||||
artifact = session.artifact(table, mode=(mode if table.speculative else None), command=server.command, notes=[
|
||||
f"pp{prompt_tokens}/tg{output_tokens}; depth is a cached conversation prefix of synthetic text "
|
||||
f"({PREFIX_REPLY_TOKENS}-token reply); measured turn task: {spec.get('workload', 'prose')}; "
|
||||
f"({PREFIX_REPLY_TOKENS}-token reply); measured turn task is recorded per row; "
|
||||
f"tolerance max(32, {fraction:.3%}) on cache_n and prompt_n; actual counts per sample in rows",
|
||||
f"synthetic text: {tokenizer.ratio:.3f} tokens/word, template overhead {tokenizer.overhead} tokens",
|
||||
*[f"not measured, {failure}" for failure in failures],
|
||||
@@ -434,6 +483,12 @@ TASKS = {
|
||||
"Write at least 500 words."),
|
||||
# Fully predictable output: the single-user analogue of the `repetition` corpus.
|
||||
"repetition": "\n\nRepeat the passage above word for word, from the beginning.",
|
||||
"copy": ("\n\nCopy ONLY the passage in this message verbatim. Start with its first word, "
|
||||
"preserve every word and punctuation mark, and continue until the complete "
|
||||
"passage has been copied. Do not comment, summarize, or use ellipses."),
|
||||
"story": ("\n\nUse the passage only as inspiration for an original story about a river town. "
|
||||
"Do not analyze or summarize the passage. Begin the story immediately and write "
|
||||
"at least 1000 words, with detailed scenes, dialogue, and a developing plot."),
|
||||
# Reasoning output: the generated tokens are chain-of-thought rather than prose.
|
||||
# Thinking is enabled per request so the cached prefix turn renders unchanged.
|
||||
"thinking": ("\n\nHow many distinct words appear in the passage above, and which three are "
|
||||
@@ -566,8 +621,15 @@ def run_multi(session: Session, table: TableSpec, display_table: TableSpec | Non
|
||||
suite_bytes = json.dumps(
|
||||
{"recipe": "single-user-d0", "task": task, "prompt": prompt},
|
||||
sort_keys=True).encode()
|
||||
if (prefill_first and users > 1 and session.target == "gufo"
|
||||
and session.reference_kind == "ds4"):
|
||||
# Keep prefill arithmetic identical to the C1 control;
|
||||
# simultaneous cold prefills use smaller scheduler chunks.
|
||||
session.request(server.base_url, prompt, 1)
|
||||
log_offset = (server.log_path.stat().st_size
|
||||
if session.target == "reference" and session.reference_kind == "ds4" else 0)
|
||||
report = run_corpus_benchmark(
|
||||
base_url=server.base_url, model=MODEL_ALIAS, cases=cases,
|
||||
base_url=server.base_url, model=session.request_model, cases=cases,
|
||||
workload_id=f"{cfg.model}-{table.id}-{session.target}-{mode}",
|
||||
max_tokens=int(spec["output_tokens"]),
|
||||
temperature=float(cfg.data["sampling"]["temperature"]),
|
||||
@@ -579,12 +641,19 @@ def run_multi(session: Session, table: TableSpec, display_table: TableSpec | Non
|
||||
cache_prompt=(True if prefill_first else
|
||||
(False if not spec.get("cache_prompt", False) else None)),
|
||||
prefill_first=prefill_first,
|
||||
pin_slots=prefill_first and session.target == "reference",
|
||||
pin_slots=prefill_first and session.target == "reference" and session.reference_kind != "ds4",
|
||||
preparation_tokens=0 if session.target == "reference" and session.reference_kind == "ds4" else 1,
|
||||
reference=reference,
|
||||
notes=[note for note in (
|
||||
session.reference_version(mode), " ".join(public_command(server.command)),
|
||||
"fresh server per concurrency level") if note],
|
||||
)
|
||||
if session.target == "reference" and session.reference_kind == "ds4":
|
||||
from ds4.server_metrics import cohort_metrics
|
||||
with server.log_path.open("rb") as log:
|
||||
log.seek(log_offset)
|
||||
cohort_metrics(log.read().decode(), report["results"][f"c{users}"],
|
||||
int(spec["output_tokens"]))
|
||||
except (RuntimeError, OSError):
|
||||
exit_code = server.failure_exit_code
|
||||
wait_process_exit(server)
|
||||
@@ -598,7 +667,7 @@ def run_multi(session: Session, table: TableSpec, display_table: TableSpec | Non
|
||||
combined.setdefault("results", {})[f"c{remaining}"] = {"unavailable": reason}
|
||||
print(f"{table.id} {session.target} {mode} C{remaining}: n/a, {reason}")
|
||||
combined["modelBench"] = {"table": table.id, "target": session.target, "mode": mode,
|
||||
"measuredOn": dt.date.today().isoformat()}
|
||||
"measuredOn": dt.datetime.now(dt.timezone.utc).date().isoformat()}
|
||||
save_artifact(path, combined)
|
||||
break
|
||||
wait_process_exit(server)
|
||||
@@ -620,6 +689,12 @@ def run_multi(session: Session, table: TableSpec, display_table: TableSpec | Non
|
||||
"tokensPerWord": tokenizer.ratio,
|
||||
"promptSha256": hashlib.sha256(prompt.encode()).hexdigest(),
|
||||
}
|
||||
result["promptPreparation"] = (
|
||||
"C1 checkpoint followed by concurrent preparation"
|
||||
if prefill_first and users > 1 and session.target == "gufo"
|
||||
and session.reference_kind == "ds4"
|
||||
else "concurrent preparation" if prefill_first else "cold prompts"
|
||||
)
|
||||
if combined is None or combined.get("artifactType") != report.get("artifactType"):
|
||||
combined = report
|
||||
else:
|
||||
@@ -628,7 +703,7 @@ def run_multi(session: Session, table: TableSpec, display_table: TableSpec | Non
|
||||
combined["workload"]["concurrency"] = [
|
||||
int(key[1:]) for key in combined["results"] if key.startswith("c")]
|
||||
combined["modelBench"] = {"table": table.id, "target": session.target, "mode": mode,
|
||||
"measuredOn": dt.date.today().isoformat()}
|
||||
"measuredOn": dt.datetime.now(dt.timezone.utc).date().isoformat()}
|
||||
save_artifact(path, combined)
|
||||
from .render import _serving_rate
|
||||
|
||||
@@ -771,4 +846,11 @@ RUNNERS = {
|
||||
|
||||
|
||||
def run_table(session: Session, table: TableSpec) -> None:
|
||||
if (session.target == "reference" and table.kind != "loading"
|
||||
and not session.config.data["reference"].get("stage_timings", True)):
|
||||
raise RuntimeError(
|
||||
f"{session.config.reference_name} does not expose qualified per-request stage timings; "
|
||||
"qualify timing and prepared-session reuse before running its throughput/memory tables. "
|
||||
"Whole-request wall time cannot substitute for decode time."
|
||||
)
|
||||
RUNNERS[table.kind](session, table)
|
||||
@@ -12,7 +12,7 @@ from .artifacts import artifact_path, load_artifact
|
||||
from .config import BenchConfig, TableSpec
|
||||
|
||||
TODO = "TODO"
|
||||
NA = "n/a" # reference cell that cannot be measured on this host (see artifacts/unavailable.json)
|
||||
NA = "N/A" # unavailable comparison (see artifacts/unavailable.json)
|
||||
MARKER_RE = re.compile(
|
||||
r"<!-- bench:(?P<id>[\w-]+) -->\n(?P<body>.*?)<!-- /bench -->", re.DOTALL
|
||||
)
|
||||
@@ -310,6 +310,15 @@ def _serving_rate(report: dict[str, Any] | None, users: int) -> float | None:
|
||||
return None
|
||||
if sum(round_.get("sampleCount", 0) for round_ in rounds) != len(samples):
|
||||
return None
|
||||
logged = result.get("loggedDecodeTimers")
|
||||
if logged:
|
||||
durations = logged.get("durations_ms", [])
|
||||
tokens = logged.get("tokens_per_request", 0)
|
||||
if (logged.get("source") != "antirez-server-log" or len(durations) != len(samples)
|
||||
or tokens <= 0 or any(sample["completion_tokens"] != tokens for sample in samples)
|
||||
or any(not math.isfinite(ms) or ms <= 0 for ms in durations)):
|
||||
return None
|
||||
return math.fsum(tokens * 1000 / ms for ms in durations) / len(rounds)
|
||||
rates = [sample.get("decode_tokens_per_second") for sample in samples]
|
||||
if any(rate is None or not math.isfinite(rate) or rate < 0 for rate in rates):
|
||||
return None
|
||||
@@ -344,9 +353,11 @@ def _pick(fresh: str | None, existing: str | None) -> str:
|
||||
|
||||
def _ref(fresh: str | None, unavailable: bool = False) -> str:
|
||||
"""Reference cells come from artifacts only, so layout changes never carry stale values."""
|
||||
if unavailable:
|
||||
return NA
|
||||
if fresh is not None:
|
||||
return fresh
|
||||
return NA if unavailable else TODO
|
||||
return TODO
|
||||
|
||||
|
||||
def unavailable_rows(config: BenchConfig, table: TableSpec) -> set[str] | None:
|
||||
@@ -389,7 +400,8 @@ def render_table(config: BenchConfig, table: TableSpec, existing: dict[str, dict
|
||||
workloads = table.workload_tables()
|
||||
if workloads:
|
||||
def best_pp(engine: str) -> str | None:
|
||||
candidates = [fresh.get(f"{engine}_pp_{w.spec['label']}") for w in workloads]
|
||||
candidates = [fresh.get(f"{engine}_pp_{w.spec['label']}") for w in workloads
|
||||
if engine != "ref" or not fresh.get(f"ref_unavailable_{w.spec['label']}")]
|
||||
return max((v for v in candidates if _number(v) is not None),
|
||||
key=_number, default=None)
|
||||
|
||||
|
||||
@@ -1145,6 +1145,7 @@ def run_corpus_benchmark(
|
||||
notes: list[str] | None = None,
|
||||
prefill_first: bool = False,
|
||||
pin_slots: bool = False,
|
||||
preparation_tokens: int = 1,
|
||||
) -> dict[str, Any]:
|
||||
if not model:
|
||||
raise ValueError("model must not be empty")
|
||||
@@ -1160,6 +1161,8 @@ def run_corpus_benchmark(
|
||||
raise ValueError("endpoint profile must be gufo or openai")
|
||||
if prefill_first and cache_prompt is not True:
|
||||
raise ValueError("prepared decoding requires cache_prompt=true")
|
||||
if preparation_tokens not in (0, 1):
|
||||
raise ValueError("prompt preparation supports zero or one output token")
|
||||
|
||||
results: dict[str, Any] = {}
|
||||
warnings: list[str] = []
|
||||
@@ -1200,17 +1203,18 @@ def run_corpus_benchmark(
|
||||
for group_index, group in enumerate(groups):
|
||||
if prefill_first:
|
||||
# Finish every prompt before releasing the measured cohort.
|
||||
# One output token leaves a reusable prompt frontier; the
|
||||
# measured request repeats the exact prompt, not its output.
|
||||
# Antirez accepts zero-token prefill; one generated token
|
||||
# would advance its state beyond the reusable frontier.
|
||||
# Gufo/llama.cpp retain a prompt checkpoint after one token.
|
||||
prepared = _run_corpus_round(
|
||||
base_url=base_url, model=model, cases=group,
|
||||
max_tokens=1, temperature=temperature,
|
||||
max_tokens=preparation_tokens, temperature=temperature,
|
||||
timeout_seconds=timeout_seconds, concurrency=concurrency,
|
||||
repetition=-(repetition + 1), group_index=group_index,
|
||||
endpoint_profile=endpoint_profile, cache_prompt=True,
|
||||
pin_slots=pin_slots,
|
||||
)
|
||||
if any(sample.completion_tokens != 1 for sample in prepared.samples):
|
||||
if any(sample.completion_tokens != preparation_tokens for sample in prepared.samples):
|
||||
raise RuntimeError("prompt preparation did not complete")
|
||||
preparations.append({
|
||||
"repetition": repetition, "group": group_index,
|
||||
@@ -1308,7 +1312,7 @@ def run_corpus_benchmark(
|
||||
else "cycle-from-start"
|
||||
),
|
||||
"warmupPolicy": "prepare-every-session" if prefill_first else "first-cohort",
|
||||
"warmupMaxOutputTokens": 1 if prefill_first else min(max_tokens, 16),
|
||||
"warmupMaxOutputTokens": preparation_tokens if prefill_first else min(max_tokens, 16),
|
||||
"maxOutputTokens": max_tokens,
|
||||
"temperature": temperature,
|
||||
"warmupRounds": warmup_rounds,
|
||||
|
||||
Reference in new issue
Block a user