mirror of
https://github.com/Niko1221/Strata.git
synced 2026-11-07 06:15:58 -10:00
web: a Model settings view for documented run config keys (#564)
The About tab's Model settings card reads GET /config and saves through POST /config: the sampling defaults, reasoning_budget_tokens, fit_max_tokens, anthropic_thinking, effort_position, aliases, idle_unload_s, lazy_load, engine_silence_s, api_monitor, open_browser and --vram-reserve-mib (serve/runconfig.py). Every value is checked first (one bad value changes nothing), only the named keys change, every other key of strata-<model>.json stays as it is (as setup run again keeps them, #629), the earlier file is kept as .bak, and the change is used from the next start. POST takes JSON from Strata's own page only, like /settings, and the API key when one is set; the network, key, MCP and program keys are not editable there.
This commit is contained in:
1 parent
13d0e2f4a3
commit
4546bf7c03
8 files changed
+481
-4
No files matched your search
+6
-4
@@ -24,15 +24,17 @@ this PC only. The details and every setting are in [docs/DETAILS.md](docs/DETAIL
|
||||
your network, and the server then warns when no API key is set.
|
||||
- **API key.** `"api_key"` in the run config (or `STRATA_API_KEY`) is required on `/v1/*` and on every endpoint
|
||||
that shows the model's state, requests or answers (`/status`, `/metrics`, `/settings`, `/mcp`, `/props`, `/slots`,
|
||||
`/api/requests`) and on every `POST`. It is compared in constant time. Set one before you open the server to your network or put a tunnel in front of it.
|
||||
`/api/requests`, `/config`) and on every `POST`. It is compared in constant time. Set one before you open the
|
||||
server to your network or put a tunnel in front of it.
|
||||
- **Host check (DNS rebinding, 0.1.38).** Without an API key the server answers only requests whose `Host` is a name
|
||||
it knows (`localhost`, an IP address, the address it listens on and, when it listens beyond this PC, this PC's
|
||||
name). Others get 403. `"allowed_hosts"` adds names; with an API key the key decides.
|
||||
- **Origin check (0.1.38).** Without an API key, a browser `POST` to `/v1/*` from another web site's page (it carries
|
||||
an `Origin` header) gets 403 unless that origin is Strata's own page, `localhost`, an allowed host, or listed in
|
||||
`"trusted_origins"` / `"cors_origins"`. Changing settings (`/settings`), `/load` and `/unload` and the MCP tools are
|
||||
accepted only as JSON from Strata's own page (or `"trusted_origins"`), so a page elsewhere cannot change settings
|
||||
or run tools.
|
||||
`"trusted_origins"` / `"cors_origins"`. Changing settings (`/settings`, and `/config`: the few run config keys the
|
||||
page's Model settings may change - never the network, key, MCP or program keys), `/load` and `/unload` and the
|
||||
MCP tools are accepted only as JSON from Strata's own page (or `"trusted_origins"`), so a page elsewhere cannot
|
||||
change settings or run tools.
|
||||
- **CORS** is off unless `"cors_origins"` lists origins, and then only for `/v1/*`.
|
||||
- **MCP tools** are opt-in: only the servers you put in the run config, only for requests from Strata's own page
|
||||
that ask for them. They run with your user's rights, and the model decides when to call them.
|
||||
|
||||
@@ -535,6 +535,14 @@ print(r.choices[0].message.content)
|
||||
- **Model aliases** (0.1.32). `"aliases": ["qwen", "local-model"]` in `strata-<model>.json` lists the model under
|
||||
those names too in `/v1/models` (each with its own `id`, and in the model's `aliases`), like llama-server's
|
||||
`--alias`; a request naming one is answered under that name. Any other name is still served, as before.
|
||||
- **Model settings in the web page (0.1.39, #564).** The About tab's Model settings card shows and changes a few of
|
||||
the keys above in `strata-<model>.json`: the `sampling` defaults (temperature, top_p, top_k, min_p),
|
||||
`reasoning_budget_tokens`, `fit_max_tokens`, `anthropic_thinking`, `effort_position`, `aliases`, `idle_unload_s`,
|
||||
`lazy_load`, `engine_silence_s`, `api_monitor`, `open_browser` and `--vram-reserve-mib`. An empty field removes the
|
||||
key (its default). Every other key of the file stays as it is, the earlier file is kept as
|
||||
`strata-<model>.json.bak`, and the model uses the change from its next start. Only Strata's own page can save
|
||||
(JSON, the API key when one is set, as for the Chat settings); the network, key, MCP and program keys are not
|
||||
editable there.
|
||||
- **From other devices on your network.** The server listens on your PC only (`127.0.0.1`) unless you say otherwise:
|
||||
run setup with `START-HERE.bat --setup --host 0.0.0.0 --api-key some-long-secret` (or add `"host": "0.0.0.0"` and
|
||||
`"api_key": "..."` to `strata-<model>.json`). The server window then prints this PC's addresses
|
||||
|
||||
@@ -0,0 +1,179 @@
|
||||
"""serve/runconfig.py - #564: the web page's Settings view of the run config (strata-<model>.json).
|
||||
|
||||
A short list of documented keys can be read and changed from Strata's own page (GET / POST /config). Everything
|
||||
else in the file - the keys setup writes, the network and security keys (host, api_key, cors_origins,
|
||||
trusted_origins, allowed_hosts), the MCP servers and before_load (which run programs), sampling keys not listed
|
||||
here - is kept as it is: a change touches only the keys it names (as setup run again keeps the user's keys, #629).
|
||||
The earlier file is kept as strata-<model>.json.bak. The server reads the config when it starts, so a change is
|
||||
used from the next start on.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
# (key, kind, help). kind: "bool", "int>=0", "num>=0", ("enum", values), "names", or ("sampling", check) for a key
|
||||
# of the "sampling" block, ("arg", flag) for an engine option kept in "args".
|
||||
EDITABLE = [
|
||||
("sampling.temperature", ("sampling", "num>=0"),
|
||||
"Default temperature for requests that send none (0 = greedy, the default without a sampling block)"),
|
||||
("sampling.top_p", ("sampling", "0<x<=1"), "Default top_p for requests that send none"),
|
||||
("sampling.top_k", ("sampling", "1..64"), "Default top_k for requests that send none (1-64)"),
|
||||
("sampling.min_p", ("sampling", "0<=x<=1"), "Default min_p for requests that send none"),
|
||||
("reasoning_budget_tokens", "int>=0", "Cap the thinking of every request at this many tokens (0 or empty: no cap)"),
|
||||
("fit_max_tokens", "bool", "Shorten a max_tokens that does not fit the context instead of answering 400"),
|
||||
("anthropic_thinking", ("enum", ["model", "on_request"]),
|
||||
"Anthropic requests that do not ask for thinking: think as the model does (model) or not (on_request)"),
|
||||
("effort_position", ("enum", ["start", "end"]),
|
||||
"Where a non-default reasoning effort goes: start (the default) or end (keeps the cache when it changes)"),
|
||||
("aliases", "names", "Other model names the server lists and answers to (comma-separated)"),
|
||||
("idle_unload_s", "num>=0", "Unload the model after this many seconds without requests (0 or empty: never)"),
|
||||
("lazy_load", "bool", "Start without loading the model; the first request loads it (text only)"),
|
||||
("engine_silence_s", "num>=0", "End a request when the engine says nothing for this long (default 300 s, 0 = wait)"),
|
||||
("api_monitor", "bool", "Keep the last 100 requests' prompts and answers in memory for /api-monitor"),
|
||||
("open_browser", "bool", "Open the chat page in the browser when the model is ready"),
|
||||
("vram_reserve_mib", ("arg", "--vram-reserve-mib"),
|
||||
"VRAM in MiB the engine leaves free for other programs (engine default 700)"),
|
||||
]
|
||||
SPEC = {k: kind for k, kind, _ in EDITABLE}
|
||||
|
||||
|
||||
def _arg(cfg: dict, flag: str):
|
||||
a = cfg.get("args") if isinstance(cfg.get("args"), list) else []
|
||||
if flag in a[:-1]:
|
||||
v = str(a[a.index(flag) + 1])
|
||||
return int(v) if v.isdigit() else v
|
||||
return None
|
||||
|
||||
|
||||
def value_of(cfg: dict, key: str):
|
||||
kind = SPEC[key]
|
||||
if isinstance(kind, tuple) and kind[0] == "sampling":
|
||||
s = cfg.get("sampling")
|
||||
return s.get(key.split(".", 1)[1]) if isinstance(s, dict) else None
|
||||
if isinstance(kind, tuple) and kind[0] == "arg":
|
||||
return _arg(cfg, kind[1])
|
||||
return cfg.get(key)
|
||||
|
||||
|
||||
def view(cfg: dict, path: str | Path) -> dict:
|
||||
"""GET /config: the editable keys with their values (None: not set, the default applies)."""
|
||||
out = []
|
||||
for key, kind, help_ in EDITABLE:
|
||||
k = kind[0] if isinstance(kind, tuple) else kind
|
||||
out.append({"key": key, "value": value_of(cfg, key), "help": help_,
|
||||
"kind": "number" if k in ("sampling", "arg", "int>=0", "num>=0") else k,
|
||||
**({"choices": kind[1]} if k == "enum" else {})})
|
||||
return {"file": Path(path).name, "keys": out,
|
||||
"note": "Saved to the run config; used from the next start of the model."}
|
||||
|
||||
|
||||
def _number(key, v, whole=False, lo=0.0, hi=None, lo_open=False):
|
||||
if isinstance(v, bool) or not isinstance(v, (int, float)) or (whole and v != int(v)) or \
|
||||
(v <= lo if lo_open else v < lo) or (hi is not None and v > hi):
|
||||
rng = f"{'more than' if lo_open else 'at least'} {lo:g}" + (f" and at most {hi:g}" if hi is not None else "")
|
||||
raise ValueError(f"{key}: expected a {'whole ' if whole else ''}number, {rng}, not {v!r}")
|
||||
return int(v) if whole else v
|
||||
|
||||
|
||||
def check(key: str, v, cfg: dict):
|
||||
"""The value to store for `key`, or a ValueError naming what is expected. None removes the key (its default)."""
|
||||
if key not in SPEC:
|
||||
raise ValueError(f"{key!r} cannot be changed here (only the keys the Settings view lists)")
|
||||
if v is None:
|
||||
return None
|
||||
kind = SPEC[key]
|
||||
k = kind[0] if isinstance(kind, tuple) else kind
|
||||
if k == "bool":
|
||||
if not isinstance(v, bool):
|
||||
raise ValueError(f"{key}: expected true or false, not {v!r}")
|
||||
if key == "lazy_load" and v and cfg.get("vision"):
|
||||
raise ValueError("lazy_load: lazy loading is text-only, and this model reads images (\"vision\")")
|
||||
return v
|
||||
if k == "enum":
|
||||
if v not in kind[1]:
|
||||
raise ValueError(f"{key}: expected one of {', '.join(kind[1])}, not {v!r}")
|
||||
return v
|
||||
if k == "names":
|
||||
if isinstance(v, str):
|
||||
v = [x.strip() for x in v.split(",") if x.strip()]
|
||||
if not isinstance(v, list) or not all(isinstance(x, str) and x.strip() and len(x) <= 128 for x in v):
|
||||
raise ValueError(f"{key}: expected a list of model names, not {v!r}")
|
||||
return v or None
|
||||
if k == "int>=0":
|
||||
return _number(key, v, whole=True)
|
||||
if k == "num>=0":
|
||||
return _number(key, v)
|
||||
if k == "arg":
|
||||
return _number(key, v, whole=True)
|
||||
rule = kind[1] # a sampling key: the server's own rules
|
||||
if rule == "num>=0":
|
||||
return float(_number(key, v))
|
||||
if rule == "0<x<=1":
|
||||
return float(_number(key, v, hi=1, lo_open=True))
|
||||
if rule == "0<=x<=1":
|
||||
return float(_number(key, v, hi=1))
|
||||
return _number(key, v, whole=True, lo=1, hi=64)
|
||||
|
||||
|
||||
def apply(cfg: dict, changes: dict) -> tuple[dict, list[str]]:
|
||||
"""The config with `changes` ({key: value or None}) made, and the keys that changed. Every value is checked
|
||||
first, so a request with one bad value changes nothing (ValueError)."""
|
||||
if not isinstance(changes, dict) or not changes:
|
||||
raise ValueError('send {"set": {"<key>": <value or null>, ...}}')
|
||||
checked = {k: check(k, v, cfg) for k, v in changes.items()}
|
||||
new = json.loads(json.dumps(cfg)) # a deep copy: the caller's dict is left alone
|
||||
changed = []
|
||||
for key, v in checked.items():
|
||||
if value_of(new, key) == v:
|
||||
continue
|
||||
kind = SPEC[key]
|
||||
if isinstance(kind, tuple) and kind[0] == "sampling":
|
||||
s = dict(new.get("sampling") or {}) if isinstance(new.get("sampling"), dict) else {}
|
||||
sub = key.split(".", 1)[1]
|
||||
if v is None:
|
||||
s.pop(sub, None)
|
||||
else:
|
||||
s[sub] = v
|
||||
if s:
|
||||
new["sampling"] = s
|
||||
else:
|
||||
new.pop("sampling", None)
|
||||
elif isinstance(kind, tuple) and kind[0] == "arg":
|
||||
a = list(new.get("args") or [])
|
||||
flag = kind[1]
|
||||
if flag in a[:-1]:
|
||||
i = a.index(flag)
|
||||
if v is None:
|
||||
del a[i:i + 2]
|
||||
else:
|
||||
a[i + 1] = str(v)
|
||||
elif v is not None:
|
||||
a += [flag, str(v)]
|
||||
new["args"] = a
|
||||
elif v is None:
|
||||
new.pop(key, None)
|
||||
else:
|
||||
new[key] = v
|
||||
changed.append(key)
|
||||
return new, changed
|
||||
|
||||
|
||||
def save(path: str | Path, cfg: dict) -> Path:
|
||||
"""The config written whole (a temporary file moved over the old one), the earlier one kept as <name>.bak."""
|
||||
path = Path(path)
|
||||
bak = path.with_name(path.name + ".bak")
|
||||
shutil.copyfile(path, bak)
|
||||
tmp = path.with_name(path.name + ".tmp")
|
||||
tmp.write_text(json.dumps(cfg, indent=1), encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
return bak
|
||||
|
||||
|
||||
def load(path: str | Path) -> dict:
|
||||
cfg = json.loads(Path(path).read_text(encoding="utf-8-sig"))
|
||||
if not isinstance(cfg, dict):
|
||||
raise ValueError(f"{Path(path).name} is not a JSON object")
|
||||
return cfg
|
||||
@@ -53,6 +53,7 @@ sys.path.insert(0, str(ROOT)) # run as a script (run-<model>.bat) as well as a
|
||||
from serve.frontend import (ChatTemplate, Event, OutputParser, anthropic_to_messages, # noqa: E402
|
||||
images_of, mark_think_literals, openai_to_messages, unmark_think_literals)
|
||||
from serve.mcp import McpCancelled, hub_from_config # noqa: E402
|
||||
from serve import runconfig # noqa: E402
|
||||
from serve.winjob import contain # noqa: E402
|
||||
from serve.structured import StructuredOutputError, prepare_format, validated_json # noqa: E402
|
||||
|
||||
@@ -1065,6 +1066,8 @@ class Service:
|
||||
# turn before the answer instead of the top of the prompt, so switching it keeps the cached conversation
|
||||
self.effort_end = False
|
||||
self.conv_log = ConvCacheLog() # #596: the parked conversations, from the engine's log
|
||||
self.config_path = None # #564: the run config the web page's Settings view edits
|
||||
self.config_lock = threading.Lock()
|
||||
# #321: browser pages of these origins may call /v1/* (CORS; "*" = any page - only with an api_key that
|
||||
# matters); empty = no CORS headers at all, as before
|
||||
self.cors_origins: list[str] = []
|
||||
@@ -2296,6 +2299,10 @@ def make_handler(svc: Service):
|
||||
if self._authorized():
|
||||
self._json(200, {"shared": bool(svc.shared), "defaults": svc.shared})
|
||||
return
|
||||
if path == "/config": # #564: the Settings view's keys of the run config
|
||||
if self._authorized():
|
||||
self._config_get()
|
||||
return
|
||||
if path == "/mcp":
|
||||
# the MCP servers, their state and tools (the web app's switch and Monitor card)
|
||||
if self._authorized():
|
||||
@@ -2365,6 +2372,9 @@ def make_handler(svc: Service):
|
||||
if path == "/settings":
|
||||
self._settings()
|
||||
return
|
||||
if path == "/config":
|
||||
self._config_post()
|
||||
return
|
||||
# JSON from Strata's own page only, as /settings: else a plain form POST from any site unloads the model
|
||||
if path in ("/unload", "/load") and not self._own_page("the model can be loaded or unloaded"):
|
||||
return
|
||||
@@ -2491,6 +2501,44 @@ def make_handler(svc: Service):
|
||||
return False
|
||||
return True
|
||||
|
||||
def _config_get(self):
|
||||
if not svc.config_path:
|
||||
self._json(404, {"error": {"message": "this server was started without a run config"}})
|
||||
return
|
||||
try:
|
||||
cfg = runconfig.load(svc.config_path)
|
||||
except (OSError, ValueError) as e:
|
||||
self._json(500, {"error": {"type": "server_error", "message": f"the run config cannot be read: {e}"}})
|
||||
return
|
||||
self._json(200, runconfig.view(cfg, svc.config_path))
|
||||
|
||||
def _config_post(self):
|
||||
"""#564: change a few documented keys of the run config - JSON from Strata's own page only, as
|
||||
/settings (the key is checked before); every other key of the file stays as it is."""
|
||||
body = self.rfile.read(int(self.headers.get("Content-Length", 0)))
|
||||
if not self._own_page("the run config can be changed"):
|
||||
return
|
||||
if not svc.config_path:
|
||||
self._json(404, {"error": {"message": "this server was started without a run config"}})
|
||||
return
|
||||
try:
|
||||
req = json.loads(body or b"{}")
|
||||
with svc.config_lock:
|
||||
cfg = runconfig.load(svc.config_path)
|
||||
new, changed = runconfig.apply(cfg, req.get("set") if isinstance(req, dict) else None)
|
||||
if changed:
|
||||
bak = runconfig.save(svc.config_path, new)
|
||||
except ValueError as e:
|
||||
self._json(400, {"error": {"type": "invalid_request_error", "message": str(e)}})
|
||||
return
|
||||
except OSError as e:
|
||||
self._json(500, {"error": {"type": "server_error", "message": f"the run config cannot be written: {e}"}})
|
||||
return
|
||||
if changed:
|
||||
print(f"[strata] the Settings view changed {', '.join(changed)} in {Path(svc.config_path).name} "
|
||||
f"(the earlier file: {bak.name}); used from the next start", flush=True)
|
||||
self._json(200, {**runconfig.view(new, svc.config_path), "changed": changed})
|
||||
|
||||
def _settings(self):
|
||||
# They change what every client gets, so only the app's own page may set them
|
||||
body = self.rfile.read(int(self.headers.get("Content-Length", 0)))
|
||||
@@ -3134,6 +3182,8 @@ def main() -> int:
|
||||
svc.gpu_index = (gpu_list(cfg) or [0])[0] # the Monitor reads the card the engine runs on (issue #51)
|
||||
svc.gpu_indices = gpu_list(cfg) # ... or every card of a layer split (issue #112)
|
||||
svc.backend = cfg.get("backend") # "hip": the AMD cards' readings come from sysfs (#301)
|
||||
if a.config:
|
||||
svc.config_path = a.config # #564: the web page's Settings view
|
||||
if a.config: # the Chat settings shared with other apps, from last time
|
||||
svc.shared_path = str(Path(a.config).with_suffix("")) + ".shared-settings.json"
|
||||
try:
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""serve/test_runconfig.py - #564: the web page's Settings view of the run config (GET / POST /config), against the
|
||||
mock engine: only the listed keys change, every other key of the file stays, a bad value changes nothing, and only
|
||||
Strata's own page may write it (JSON, no foreign Origin, the API key when one is set).
|
||||
|
||||
python -m unittest serve.test_runconfig
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
import tempfile
|
||||
import unittest
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from serve import runconfig
|
||||
from serve.frontend import ChatTemplate
|
||||
from serve.server import ByteTokenizer, MockEngine, Service, serve
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
CFG = {"exe": "engine/strata.exe", "args": ["--pack", "data/packs/q2_0", "--kv", "int8", "--spec-min-p", "0.5"],
|
||||
"model_name": "qwen3.8-flash-next-q2_0", "port": 8080, "host": "0.0.0.0", "api_key": "",
|
||||
"sampling": {"temperature": 0.7, "presence_penalty": 1.5},
|
||||
"mcp_servers": {"files": {"command": "npx", "args": ["-y", "server-filesystem", "."]}},
|
||||
"cors_origins": ["http://localhost:3000"]}
|
||||
|
||||
|
||||
class Apply(unittest.TestCase):
|
||||
def test_only_the_named_keys_change(self):
|
||||
new, changed = runconfig.apply(CFG, {"sampling.top_p": 0.95, "reasoning_budget_tokens": 4096,
|
||||
"vram_reserve_mib": 2048, "aliases": "qwen, local", "lazy_load": True})
|
||||
self.assertEqual(changed, ["sampling.top_p", "reasoning_budget_tokens", "vram_reserve_mib", "aliases",
|
||||
"lazy_load"])
|
||||
self.assertEqual(new["sampling"], {"temperature": 0.7, "presence_penalty": 1.5, "top_p": 0.95})
|
||||
self.assertEqual(new["args"][-2:], ["--vram-reserve-mib", "2048"])
|
||||
self.assertEqual(new["aliases"], ["qwen", "local"])
|
||||
for k in ("exe", "model_name", "port", "host", "api_key", "mcp_servers", "cors_origins"):
|
||||
self.assertEqual(new[k], CFG[k], k)
|
||||
self.assertEqual(CFG["sampling"], {"temperature": 0.7, "presence_penalty": 1.5}) # the input is left alone
|
||||
|
||||
def test_null_is_the_default_again(self):
|
||||
cfg, _ = runconfig.apply(CFG, {"vram_reserve_mib": 2048, "fit_max_tokens": True})
|
||||
new, changed = runconfig.apply(cfg, {"vram_reserve_mib": None, "fit_max_tokens": None,
|
||||
"sampling.temperature": None, "idle_unload_s": None})
|
||||
self.assertEqual(changed, ["vram_reserve_mib", "fit_max_tokens", "sampling.temperature"])
|
||||
self.assertEqual(new["args"], CFG["args"])
|
||||
self.assertNotIn("fit_max_tokens", new)
|
||||
self.assertEqual(new["sampling"], {"presence_penalty": 1.5})
|
||||
new, _ = runconfig.apply({"sampling": {"top_k": 20}}, {"sampling.top_k": None})
|
||||
self.assertNotIn("sampling", new) # an empty block goes
|
||||
|
||||
def test_refused(self):
|
||||
for changes in ({"api_key": "x"}, {"host": "0.0.0.0"}, {"mcp_servers": {}}, {"before_load": "calc.exe"},
|
||||
{"args": []}, {"exe": "evil.exe"}, {"trusted_origins": ["*"]}, {"allowed_hosts": ["*"]},
|
||||
{"sampling.temperature": -1}, {"sampling.top_p": 0}, {"sampling.top_k": 65},
|
||||
{"sampling.min_p": 2}, {"reasoning_budget_tokens": 1.5}, {"fit_max_tokens": "yes"},
|
||||
{"effort_position": "middle"}, {"vram_reserve_mib": -5}, {"idle_unload_s": True},
|
||||
{"aliases": [1]}, {}, None, ["x"]):
|
||||
with self.subTest(changes=changes), self.assertRaises(ValueError):
|
||||
runconfig.apply(CFG, changes)
|
||||
with self.assertRaises(ValueError): # one bad value: nothing is applied
|
||||
runconfig.apply(CFG, {"fit_max_tokens": True, "sampling.top_k": 0})
|
||||
with self.assertRaises(ValueError):
|
||||
runconfig.apply({**CFG, "vision": {"gpu": True}}, {"lazy_load": True})
|
||||
|
||||
def test_view(self):
|
||||
v = runconfig.view({**CFG, "args": CFG["args"] + ["--vram-reserve-mib", "1500"]}, "x/strata-q2_0.json")
|
||||
self.assertEqual(v["file"], "strata-q2_0.json")
|
||||
got = {k["key"]: k for k in v["keys"]}
|
||||
self.assertEqual(got["sampling.temperature"]["value"], 0.7)
|
||||
self.assertEqual(got["vram_reserve_mib"]["value"], 1500)
|
||||
self.assertIsNone(got["fit_max_tokens"]["value"])
|
||||
self.assertEqual(got["anthropic_thinking"]["choices"], ["model", "on_request"])
|
||||
self.assertNotIn("api_key", got)
|
||||
self.assertNotIn("mcp_servers", got)
|
||||
|
||||
|
||||
class Http(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.dir = tempfile.TemporaryDirectory()
|
||||
self.path = Path(self.dir.name) / "strata-q2_0.json"
|
||||
self.path.write_text(json.dumps(CFG, indent=1), encoding="utf-8")
|
||||
tok = ByteTokenizer()
|
||||
self.svc = Service(MockEngine(tok, "ok", max_context=4096), tok, ChatTemplate(ROOT / "serve/chat_template.jinja"))
|
||||
self.svc.config_path = str(self.path)
|
||||
self.httpd = serve(self.svc, port=0)
|
||||
self.host = f"127.0.0.1:{self.httpd.server_address[1]}"
|
||||
|
||||
def tearDown(self):
|
||||
self.httpd.shutdown()
|
||||
self.httpd.server_close()
|
||||
self.dir.cleanup()
|
||||
|
||||
def call(self, method, body=None, headers=None):
|
||||
h = {"Content-Type": "application/json", **(headers or {})}
|
||||
req = urllib.request.Request(f"http://{self.host}/config", method=method, headers=h,
|
||||
data=None if body is None else (body if isinstance(body, bytes)
|
||||
else json.dumps(body).encode()))
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=10) as r:
|
||||
return r.status, json.loads(r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
with e:
|
||||
return e.code, json.loads(e.read())
|
||||
|
||||
def saved(self):
|
||||
return json.loads(self.path.read_text(encoding="utf-8"))
|
||||
|
||||
def test_read_and_write_from_the_page(self):
|
||||
status, v = self.call("GET")
|
||||
self.assertEqual(status, 200, v)
|
||||
self.assertEqual({k["key"]: k["value"] for k in v["keys"]}["sampling.temperature"], 0.7)
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
status, v = self.call("POST", {"set": {"sampling.temperature": 1.0, "open_browser": False}},
|
||||
{"Origin": f"http://{self.host}"})
|
||||
self.assertEqual(status, 200, v)
|
||||
self.assertEqual(v["changed"], ["sampling.temperature", "open_browser"])
|
||||
cfg = self.saved()
|
||||
self.assertEqual(cfg["sampling"]["temperature"], 1.0)
|
||||
self.assertIs(cfg["open_browser"], False)
|
||||
for k in ("mcp_servers", "cors_origins", "host", "args", "exe"):
|
||||
self.assertEqual(cfg[k], CFG[k], k)
|
||||
self.assertEqual(json.loads((self.path.parent / "strata-q2_0.json.bak").read_text(encoding="utf-8")), CFG)
|
||||
status, v = self.call("POST", {"set": {"open_browser": False}}) # no Origin (a script on this PC): allowed
|
||||
self.assertEqual((status, v["changed"]), (200, []))
|
||||
|
||||
def test_only_from_strata_s_own_page(self):
|
||||
before = self.path.read_bytes()
|
||||
status, _ = self.call("POST", {"set": {"fit_max_tokens": True}}, {"Origin": "https://evil.example"})
|
||||
self.assertEqual(status, 403)
|
||||
status, _ = self.call("POST", b'{"set": {"fit_max_tokens": true}}', {"Content-Type": "text/plain"})
|
||||
self.assertEqual(status, 415)
|
||||
for body in ({"set": {"api_key": ""}}, {"set": {"mcp_servers": {"x": {"command": "calc"}}}},
|
||||
{"set": {"sampling.top_k": 0}}, {"nothing": 1}, b"not json"):
|
||||
status, b = self.call("POST", body, {"Origin": f"http://{self.host}"})
|
||||
self.assertEqual(status, 400, (body, b))
|
||||
self.assertEqual(self.path.read_bytes(), before) # nothing was written
|
||||
self.assertFalse((self.path.parent / "strata-q2_0.json.bak").exists())
|
||||
|
||||
def test_the_api_key(self):
|
||||
self.svc.api_key = "secret"
|
||||
self.assertEqual(self.call("GET")[0], 401)
|
||||
self.assertEqual(self.call("POST", {"set": {"fit_max_tokens": True}})[0], 401)
|
||||
with contextlib.redirect_stdout(io.StringIO()):
|
||||
status, _ = self.call("POST", {"set": {"fit_max_tokens": True}}, {"Authorization": "Bearer secret"})
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIs(self.saved()["fit_max_tokens"], True)
|
||||
|
||||
def test_without_a_run_config(self):
|
||||
self.svc.config_path = None
|
||||
self.assertEqual(self.call("GET")[0], 404)
|
||||
self.assertEqual(self.call("POST", {"set": {"fit_max_tokens": True}})[0], 404)
|
||||
|
||||
def test_the_page_has_the_view(self):
|
||||
web = ROOT / "serve" / "web"
|
||||
html, js = (web / "index.html").read_text(encoding="utf-8"), (web / "app.js").read_text(encoding="utf-8")
|
||||
for el in ("cfg-card", "cfg-form", "cfg-save", "cfg-msg"):
|
||||
self.assertIn(f'id="{el}"', html)
|
||||
self.assertIn('fetch("config", {method: "POST", headers: headers(true)', js)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -121,6 +121,12 @@ main { flex: 1; min-height: 0; display: flex; }
|
||||
.cc-note { margin: 0; }
|
||||
.cc-note:empty { display: none; }
|
||||
.mcp-card { display: flex; flex-direction: column; gap: var(--st-s-3); }
|
||||
.cfg-form { display: grid; grid-template-columns: minmax(180px, max-content) minmax(0, 1fr); gap: var(--st-s-3) var(--st-s-4);
|
||||
align-items: center; }
|
||||
.cfg-form label { font-size: var(--st-fs-sm); font-weight: var(--st-fw-medium); }
|
||||
.cfg-form label code { font: 400 12px var(--st-font-mono); color: var(--st-ink-muted); display: block; }
|
||||
.cfg-form .st-input { height: 38px; width: 100%; max-width: 360px; }
|
||||
.cfg-actions { display: flex; align-items: center; gap: var(--st-s-3); }
|
||||
.mcp-server { display: flex; flex-direction: column; gap: var(--st-s-2); padding-top: var(--st-s-3); border-top: 1px solid var(--st-line-soft); }
|
||||
.mcp-server:first-child { padding-top: 0; border-top: 0; }
|
||||
.mcp-server__head { display: flex; align-items: center; gap: var(--st-s-2); flex-wrap: wrap; font-size: var(--st-fs-sm); }
|
||||
@@ -161,6 +167,8 @@ main { flex: 1; min-height: 0; display: flex; }
|
||||
.metrics { grid-template-columns: repeat(2, minmax(0, 1fr)); }
|
||||
.monitor-row { grid-template-columns: 1fr; }
|
||||
.cc-bars { grid-template-columns: 1fr; }
|
||||
.cfg-form { grid-template-columns: 1fr; gap: 4px; }
|
||||
.cfg-form .st-input { margin-bottom: var(--st-s-2); }
|
||||
}
|
||||
@media (max-width: 640px) {
|
||||
.st-header { padding: 0 var(--st-s-3); gap: var(--st-s-2); }
|
||||
|
||||
@@ -79,6 +79,7 @@ function showTab(name) {
|
||||
if (location.hash.slice(1) !== tab) history.replaceState(null, "", tab === "chat" ? location.pathname : `#${tab}`);
|
||||
if (tab === "chat") $("input").focus();
|
||||
if (tab === "monitor") loadMcp();
|
||||
if (tab === "about") loadConfig();
|
||||
if (lastMetrics) render(lastMetrics);
|
||||
}
|
||||
for (const b of document.querySelectorAll(".st-tab")) b.onclick = () => showTab(b.dataset.tab);
|
||||
@@ -428,6 +429,62 @@ function renderMcp() {
|
||||
}).join("");
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------ Model settings (GET / POST /config, #564)
|
||||
// A few documented keys of the run config (strata-<model>.json), for every client, from the next start on. The
|
||||
// server lists them, checks every value and keeps every other key of the file as it is.
|
||||
let cfgKeys = [];
|
||||
async function loadConfig() {
|
||||
let r;
|
||||
try { r = await fetch("config", {headers: headers()}); } catch (e) { return; }
|
||||
if (!r.ok) { $("cfg-card").hidden = true; return; } // no run config, an older server, or no key yet
|
||||
const c = await r.json();
|
||||
cfgKeys = c.keys || [];
|
||||
$("cfg-file").textContent = c.file || "";
|
||||
$("cfg-form").innerHTML = cfgKeys.map((k, i) => {
|
||||
const id = `cfg-${i}`, v = k.value;
|
||||
let input;
|
||||
if (k.kind === "bool" || k.kind === "enum") {
|
||||
const opts = k.kind === "bool" ? [["true", "on"], ["false", "off"]] : k.choices.map((x) => [x, x]);
|
||||
const cur = v == null ? "" : String(v);
|
||||
input = `<select class="st-input" id="${id}"><option value=""${cur === "" ? " selected" : ""}>default</option>` +
|
||||
opts.map(([val, text]) => `<option value="${esc(val)}"${cur === val ? " selected" : ""}>${esc(text)}</option>`).join("") + `</select>`;
|
||||
} else {
|
||||
const text = v == null ? "" : Array.isArray(v) ? v.join(", ") : String(v);
|
||||
input = `<input class="st-input" id="${id}" ${k.kind === "number" ? 'type="number" step="any" min="0"' : 'type="text"'} ` +
|
||||
`value="${esc(text)}" placeholder="default" autocomplete="off">`;
|
||||
}
|
||||
return `<label for="${id}" title="${esc(k.help)}">${esc(k.help)}<code>${esc(k.key)}</code></label>${input}`;
|
||||
}).join("");
|
||||
$("cfg-card").hidden = false;
|
||||
}
|
||||
function configValue(k, el) {
|
||||
const s = el.value.trim();
|
||||
if (s === "") return null;
|
||||
if (k.kind === "bool") return s === "true";
|
||||
if (k.kind === "number") return Number(s);
|
||||
return s; // enum, or names (the server splits them at commas)
|
||||
}
|
||||
$("cfg-save").addEventListener("click", async () => {
|
||||
const set = {};
|
||||
cfgKeys.forEach((k, i) => {
|
||||
const v = configValue(k, $(`cfg-${i}`));
|
||||
const old = Array.isArray(k.value) ? k.value.join(", ") : k.value;
|
||||
if (JSON.stringify(v) !== JSON.stringify(old ?? null)) set[k.key] = v;
|
||||
});
|
||||
if (!Object.keys(set).length) { $("cfg-msg").textContent = "Nothing changed."; return; }
|
||||
try {
|
||||
const r = await fetch("config", {method: "POST", headers: headers(true), body: JSON.stringify({set})});
|
||||
const b = await r.json();
|
||||
if (!r.ok) throw new Error((b.error || {}).message || `HTTP ${r.status}`);
|
||||
$("cfg-msg").textContent = b.changed.length
|
||||
? `Saved (${b.changed.join(", ")}); the earlier file is ${b.file}.bak. Start the model again to use it.` : "Nothing changed.";
|
||||
loadConfig();
|
||||
} catch (e) {
|
||||
$("cfg-msg").textContent = "";
|
||||
toast("error", "Not saved", String(e.message || e), 6000);
|
||||
}
|
||||
});
|
||||
|
||||
// ------------------------------------------------------------------ Markdown (escaped first, then formatted)
|
||||
function inline(s) {
|
||||
const codes = [];
|
||||
|
||||
@@ -173,6 +173,14 @@
|
||||
<p class="muted small">Chats, settings and the key are kept in this browser only.
|
||||
<a href="https://github.com/Niko1221/Strata" target="_blank" rel="noopener noreferrer">Strata on GitHub</a></p>
|
||||
</div>
|
||||
<div class="st-card cfg-card" id="cfg-card" hidden>
|
||||
<div class="req-head"><span class="card-title">Model settings</span><span class="muted small" id="cfg-file"></span></div>
|
||||
<p class="muted small">Kept in the model's run config for every client. An empty field is the default. They
|
||||
take effect the next time the model starts (close Strata's window and start it again).</p>
|
||||
<form class="cfg-form" id="cfg-form"></form>
|
||||
<div class="cfg-actions"><button class="st-btn" id="cfg-save" type="button">Save</button>
|
||||
<span class="muted small" id="cfg-msg" role="status"></span></div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
Reference in new issue
Block a user