cloudstack-llc/mlx-knife
1
0
Fork 0
mirror of https://github.com/cloudstack-llc/mlx-knife.git synced 2026-07-19 14:43:36 -04:00
Code Issues Packages Projects Releases Wiki Activity

chore: custom build

Browse Source
This commit is contained in:
Ashok Gelal
2025-09-30 15:40:58 -04:00
parent 42fd195095
commit c7b5cc56fa
4 changed files with 783 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ELECTRON_EXTERNAL_PYTHON.md
+142
View File
@@ -0,0 +1,142 @@
# External Python Mode (Electron-embedded Python)
This variant skips PyInstaller. We vendor all Python dependencies (and `mlxk2` itself) into a folder and run the CLI with the Python interpreter you bundle inside the Electron app (`MyApp.app/Contents/Resources/python/bin/python3`).
## Build quick reference
```bash
# Preferred: install using the embedded Python
EMBEDDED_PYTHON="/path/to/MyApp.app/Contents/Resources/python/bin/python3" \
scripts/build_external_python.sh
# Dev fallback (uses ./python/bin/python3 if present, else system python3)
scripts/build_external_python.sh
# Provide custom MLX / MLX-LM wheels
scripts/build_external_python.sh \
--mlx-wheel scripts/mlx-custom.whl \
--mlx-lm-wheel scripts/mlx-lm-custom.whl
# Wheels-only mode (no interpreter execution)
TARGET_PY_VERSION=3.12 \
VENDOR_VIA_WHEELS=1 \
scripts/build_external_python.sh
```
Output: `dist-ext-python/<BIN_NAME>/` (default `msty-mlx-studio/`), containing:
- `_vendor/` – vendored site-packages (deps + project)
- `msty-mlx-studio` – launcher that runs `-m mlxk2.cli` with `PYTHONPATH=_vendor`
### Build options reference
- `EMBEDDED_PYTHON` – interpreter used for installs (falls back to `./python/bin/python3`, then system python3).
- `MLX_WHEEL`, `MLX_LM_WHEEL`, `--mlx-wheel`, `--mlx-lm-wheel` – custom package wheels; defaults auto-detect `scripts/mlx-custom.whl` and `scripts/mlx-lm-custom.whl` if present.
- `VERSION` – temporary override for `mlxk2.__version__` while bundling; README references adjust automatically.
- `DIST_DIR`, `BIN_NAME` – output location/name (default `dist-ext-python/msty-mlx-studio`).
- `CLEAN=1` – delete the output directory before writing.
- `PIP_ARGS="..."` – extra pip flags (shared by install/download steps).
- `VENDOR_VIA_WHEELS=1`, `TARGET_PY_VERSION`, `TARGET_PLATFORM`, `TARGET_ABI` – download wheels without executing the embedded interpreter.
### Wheels-only mode (Gatekeeper-friendly)
If macOS blocks your embedded Python during the build, set `VENDOR_VIA_WHEELS=1` and choose a compatible `TARGET_PY_VERSION`. The script uses the system python to `pip download` macOS arm64 wheels and unpacks them into `_vendor`. Ensure the chosen Python minor version matches your embedded interpreter so compiled wheels load correctly.
### Custom MLX / MLX-LM wheels
Provide custom builds to keep the MLX stack in sync:
```bash
scripts/build_external_python.sh \
--mlx-wheel /abs/path/mlx.whl \
--mlx-lm-wheel /abs/path/mlx_lm.whl
```
Both options can also be set via environment variables. The script filters `mlx` and `mlx-lm` out of `requirements.txt`, then installs your wheels last so they override PyPI pins in `_vendor`.
### Diagnostics
After building, run the launcher with these flags to confirm what was bundled:
```bash
./dist-ext-python/msty-mlx-studio/msty-mlx-studio --python-info # interpreter summary
./dist-ext-python/msty-mlx-studio/msty-mlx-studio --mlx-info # mlx version + dist-info path
./dist-ext-python/msty-mlx-studio/msty-mlx-studio --pkg-info mlx_lm
./dist-ext-python/msty-mlx-studio/msty-mlx-studio --check-mlx-stack
```
`--check-mlx-stack` imports both `mlx` and `mlx_lm` to ensure API compatibility—run this after swapping in custom wheels.
### Version override
Stamp the vendored package with a release number:
```bash
VERSION=2.0.0b4 scripts/build_external_python.sh
```
The script temporarily rewrites `mlxk2/__version__` and README references for the build and restores them afterward, so your working tree stays clean.
## Packaging into Electron
Place the output under app resources, for example:
```
MyApp.app/Contents/Resources/
python/bin/python3 # bundled interpreter
mlxk2/
_vendor/
msty-mlx-studio # launcher
```
### Option A – use the launcher
- Command: `join(process.resourcesPath, 'mlxk2', 'msty-mlx-studio')`
- Env: set `RESOURCES_PATH=process.resourcesPath` (the launcher uses it to locate `python/bin/python3`).
```js
import { spawn } from 'child_process';
import { join } from 'path';
import { app } from 'electron';
const launcher = join(process.resourcesPath, 'mlxk2', 'msty-mlx-studio');
export function startServer({ host = '127.0.0.1', port = 8000, maxTokens } = {}) {
const args = ['serve', '--host', host, '--port', String(port)];
if (maxTokens != null) args.push('--max-tokens', String(maxTokens));
const env = { ...process.env, RESOURCES_PATH: process.resourcesPath };
const child = spawn(launcher, args, { env, stdio: ['ignore', 'inherit', 'inherit'] });
return child;
}
```
### Option B – call Python directly
- Python: `join(process.resourcesPath, 'python', 'bin', 'python3')`
- Vendor: `join(process.resourcesPath, 'mlxk2', '_vendor')`
- Env: `PYTHONPATH=<vendor>[:$PYTHONPATH]`, `PYTHONNOUSERSITE=1`
```js
import { spawn } from 'child_process';
import { join } from 'path';
import { app } from 'electron';
export function startServer({ host = '127.0.0.1', port = 8000, maxTokens } = {}) {
const pythonPath = join(process.resourcesPath, 'python', 'bin', 'python3');
const vendor = join(process.resourcesPath, 'mlxk2', '_vendor');
const args = ['-m', 'mlxk2.cli', 'serve', '--host', host, '--port', String(port)];
if (maxTokens != null) args.push('--max-tokens', String(maxTokens));
const env = { ...process.env, PYTHONNOUSERSITE: '1', PYTHONPATH: vendor };
const child = spawn(pythonPath, args, { env, stdio: ['ignore', 'inherit', 'inherit'] });
return child;
}
```
## Notes
- Ship a compatible macOS arm64 Python with required system libs.
- For private/offline wheels, pass `PIP_ARGS="--no-index --find-links /path/to/wheels"`.
- Clear Gatekeeper quarantine on the embedded Python during development with `scripts/clear_quarantine_python.sh /path/to/Resources/python`.
- Sanity-check at runtime:
`PYTHONPATH=<vendor> <embedded-python> -c "import mlxk2, fastapi, uvicorn, mlx, mlx_lm; print('ok')"`
- Sign + notarize the final `.app` for distribution.
docs/ELECTRON_PACKAGING.md
+97
View File
@@ -36,6 +36,19 @@ USE_HF_XET=0 scripts/build.sh
The script creates a virtualenv, installs runtime deps from `requirements.txt` (`mlx`, `mlx-lm`, `fastapi`, `uvicorn`, etc.) plus the project itself, then builds a PyInstaller bundle and packages it into `artifacts/<name>.tgz`.
### Build options reference
Environment variables / flags:
- `VERSION` – temporary override for `mlxk2.__version__` while bundling.
- `BIN_NAME` – output folder/binary name (default `msty-mlx-studio`).
- `ONEFILE=1` – build a single-file PyInstaller binary (defaults to one-dir).
- `USE_HF_XET=0` – skip the optional Hugging Face Xet plugin.
- `PYTHON_EXE=/abs/path/python3` – force a specific interpreter for the build venv.
- `MLX_WHEEL=/abs/path/mlx.whl` or `--mlx-wheel` – install a custom `mlx` wheel after deps.
- `MLX_LM_WHEEL=/abs/path/mlx_lm.whl` or `--mlx-lm-wheel` – install a matching custom `mlx-lm` wheel.
- `PIP_ARGS="..."` – forwarded to pip install commands if you need custom indexes.
Quick test after build:
```bash
@@ -65,6 +78,90 @@ Version reporting:
- No upstream source is modified. All adjustments happen in the generated entry shim used only at build time.
- The previous v1 helper worker is no longer built or packaged.
### Custom Python runtime
To freeze with a specific Python (and embed that version in the bundle), place it at `python/bin/python3` in the repo. The build script will automatically prefer this interpreter for creating the build venv and running PyInstaller. Alternatively, set `PYTHON_EXE=/absolute/path/to/python3` to override.
#### Custom MLX / MLX-LM wheels
If you need to ship custom builds:
```bash
# Auto-detected defaults: scripts/mlx-custom.whl, scripts/mlx-lm-custom.whl
MLX_WHEEL=/abs/path/mlx.whl \
MLX_LM_WHEEL=/abs/path/mlx_lm.whl \
scripts/build.sh
# Equivalent flag form
scripts/build.sh \
--mlx-wheel /abs/path/mlx.whl \
--mlx-lm-wheel /abs/path/mlx_lm.whl
```
Notes:
- `mlx` and `mlx-lm` are filtered out of `requirements.txt`; your wheels install last and override PyPI pins.
- Provide compatible builds (matching Python ABI / MLX API surface) to avoid runtime mismatches.
- Use `--check-mlx-stack` (see below) on the finished binary to confirm both packages import cleanly.
### Version override
Set `VERSION` when invoking the build to stamp the bundle with a release number:
```bash
VERSION=2.0.0b4 scripts/build.sh
```
Details:
- The script temporarily rewrites `mlxk2/__version__` (and related README references) during the build.
- Files are restored to their original contents after the script exits, so your working tree stays clean.
Steps:
- Put your desired macOS arm64 Python at `python/bin/python3`.
- Run `scripts/build.sh`.
- Verify the embedded Python in the resulting binary:
```bash
./dist/msty-mlx-studio/msty-mlx-studio --python-info
# or
MLXK2_PY_INFO=1 ./dist/msty-mlx-studio/msty-mlx-studio --version
```
This prints JSON like:
```
{
"python_version": "3.12.5",
"executable": "/path/to/dist/msty-mlx-studio/msty-mlx-studio",
"prefix": "/path/to/dist/msty-mlx-studio",
"base_prefix": null,
"frozen": true,
"meipass": "/path/to/_MEIxxxx",
"platform": "darwin"
}
```
Notes:
- `python_version` confirms which interpreter is embedded. `executable` is the frozen launcher (expected for PyInstaller).
- Ensure your chosen Python is compatible with the pinned PyInstaller (>= 6.6). For very new Python releases, update PyInstaller if needed.
### Quick diagnostics on the output bundle
- `./dist/<name>/<name> --python-info` – JSON summary of the embedded interpreter.
- `./dist/<name>/<name> --mlx-info` – JSON showing `mlx` version, module path, and dist-info directory.
- `./dist/<name>/<name> --pkg-info mlx_lm` – same diagnostic for `mlx-lm`.
- `./dist/<name>/<name> --check-mlx-stack` – verifies both `mlx` and `mlx_lm` import successfully.
Use these after applying custom wheels to ensure the expected versions were bundled.
### Clearing quarantine during development
macOS Gatekeeper may block unsigned interpreters inside the bundle. During local builds/tests, run:
```bash
scripts/clear_quarantine_python.sh dist/<name>/python
```
Or target your Electron resources path. This recurses through the folder and removes the `com.apple.quarantine` extended attribute, reporting any leftovers. (Sign and notarize for production distribution.)
## Runtime Integration
- Our Electron app handles model downloads directly (see `ELECTRON_INTEGRATION_GUIDE.md`).
scripts/build.sh
+275 -22
View File
@@ -18,6 +18,107 @@ set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"
# -----------------------------------------------------------------------------
# Temporary version override (VERSION env)
# -----------------------------------------------------------------------------
ORIG_VERSION="$(python3 - <<'PY'
import pathlib, re
text = pathlib.Path("mlxk2/__init__.py").read_text()
m = re.search(r'__version__\s*=\s*"([^"]+)"', text)
print(m.group(1) if m else "0.0.0")
PY
)"
TARGET_VERSION="${VERSION:-$ORIG_VERSION}"
TMP_INIT=""
TMP_README=""
RESTORE_VERSION=0
cleanup_version() {
if [[ "$RESTORE_VERSION" == "1" ]]; then
if [[ -n "$TMP_INIT" && -f "$TMP_INIT" ]]; then
cp "$TMP_INIT" mlxk2/__init__.py
rm -f "$TMP_INIT"
fi
if [[ -n "$TMP_README" && -f "$TMP_README" ]]; then
cp "$TMP_README" README.md
rm -f "$TMP_README"
fi
fi
}
trap cleanup_version EXIT
if [[ "$TARGET_VERSION" != "$ORIG_VERSION" ]]; then
echo "[version] Updating project version: $ORIG_VERSION -> $TARGET_VERSION"
TMP_INIT="$(mktemp)"
cp mlxk2/__init__.py "$TMP_INIT"
if [[ -f README.md ]]; then
TMP_README="$(mktemp)"
cp README.md "$TMP_README"
fi
RESTORE_VERSION=1
python3 - "$ORIG_VERSION" "$TARGET_VERSION" <<'PY'
import sys
import pathlib
import re
old, new = sys.argv[1:3]
def tag(ver: str) -> str:
if 'b' in ver:
base, beta = ver.split('b', 1)
if beta and beta.isdigit():
return f"v{base}-beta.{beta}"
return f"v{ver}"
init_path = pathlib.Path("mlxk2/__init__.py")
text = init_path.read_text()
text = re.sub(r'__version__\s*=\s*".*?"', f'__version__ = "{new}"', text)
init_path.write_text(text)
readme_path = pathlib.Path("README.md")
if readme_path.exists():
readme = readme_path.read_text()
readme = readme.replace(old, new)
readme = readme.replace(tag(old), tag(new))
readme_path.write_text(readme)
PY
fi
# -----------------------------------------------------------------------------
# CLI flags (custom wheels)
# -----------------------------------------------------------------------------
while [[ $# -gt 0 ]]; do
case "$1" in
--mlx-wheel)
shift
if [[ -n "${1:-}" ]]; then
export MLX_WHEEL="${1:-}"
shift || true
else
echo "ERROR: --mlx-wheel requires a path" >&2
exit 1
fi
;;
--mlx-lm-wheel)
shift
if [[ -n "${1:-}" ]]; then
export MLX_LM_WHEEL="${1:-}"
shift || true
else
echo "ERROR: --mlx-lm-wheel requires a path" >&2
exit 1
fi
;;
*)
echo "Unknown flag: $1" >&2
exit 1
;;
esac
done
# -----------------------------------------------------------------------------
# Platform checks
# -----------------------------------------------------------------------------
if [[ "$(uname -s)" != "Darwin" ]]; then
echo "This script must run on macOS." >&2
exit 1
@@ -35,26 +136,99 @@ SPEC_DIR="$ROOT_DIR/.pyi-spec"
mkdir -p "$ART_DIR" "$SPEC_DIR"
# -----------------------------------------------------------------------------
# Create build virtualenv
# -----------------------------------------------------------------------------
echo "[1/3] Creating build venv..."
python3 -m venv "$BUILD_VENV"
CUSTOM_PY_DIR="$ROOT_DIR/python"
if [[ -n "${PYTHON_EXE:-}" ]]; then
echo "Using override Python (PYTHON_EXE): $PYTHON_EXE"
elif [[ -x "$CUSTOM_PY_DIR/bin/python3" ]]; then
PYTHON_EXE="$CUSTOM_PY_DIR/bin/python3"
echo "Using project Python: $PYTHON_EXE"
else
PYTHON_EXE="$(command -v python3)"
echo "Using system Python: $PYTHON_EXE"
fi
"$PYTHON_EXE" -m venv "$BUILD_VENV"
source "$BUILD_VENV/bin/activate"
if ! python -m pip --version >/dev/null 2>&1; then
echo "Bootstrapping pip via ensurepip..."
python -m ensurepip --upgrade || true
fi
python -m pip install --upgrade pip setuptools wheel
# -----------------------------------------------------------------------------
# Install dependencies and project
# -----------------------------------------------------------------------------
echo "[2/3] Installing project, deps and build tools..."
# Install runtime deps for MLX-Knife 2.x
if [[ -f requirements.txt ]]; then
python -m pip install -r requirements.txt
CUSTOM_MLX_WHEEL="${MLX_WHEEL:-}"
if [[ -z "$CUSTOM_MLX_WHEEL" && -f "$ROOT_DIR/scripts/mlx-custom.whl" ]]; then
CUSTOM_MLX_WHEEL="$ROOT_DIR/scripts/mlx-custom.whl"
fi
# Install the project itself (exposes version, console entry, etc.)
python -m pip install .
if [[ -n "$CUSTOM_MLX_WHEEL" ]]; then
echo "[opt] Using custom MLX wheel: $CUSTOM_MLX_WHEEL"
[[ -f "$CUSTOM_MLX_WHEEL" ]] || { echo "ERROR: MLX wheel not found: $CUSTOM_MLX_WHEEL" >&2; exit 1; }
fi
CUSTOM_MLX_LM_WHEEL="${MLX_LM_WHEEL:-}"
if [[ -z "$CUSTOM_MLX_LM_WHEEL" && -f "$ROOT_DIR/scripts/mlx-lm-custom.whl" ]]; then
CUSTOM_MLX_LM_WHEEL="$ROOT_DIR/scripts/mlx-lm-custom.whl"
fi
if [[ -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
echo "[opt] Using custom MLX-LM wheel: $CUSTOM_MLX_LM_WHEEL"
[[ -f "$CUSTOM_MLX_LM_WHEEL" ]] || { echo "ERROR: MLX-LM wheel not found: $CUSTOM_MLX_LM_WHEEL" >&2; exit 1; }
fi
if [[ -f requirements.txt ]]; then
if [[ -n "$CUSTOM_MLX_WHEEL" || -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
REQ_TMP="$(mktemp)"
awk 'BEGIN{IGNORECASE=0} {
line=$0; trimmed=line; gsub(/^[ \t]+/,"",trimmed);
if (trimmed ~ /^#/ || trimmed ~ /^$/) { print line; next }
if (ENVIRON["FILTER_MLX"] && trimmed ~ /^mlx([ \t]|[><=]|$)/) { next }
if (ENVIRON["FILTER_MLX_LM"] && trimmed ~ /^mlx-lm([ \t]|[><=]|$)/) { next }
print line
}' FILTER_MLX=$([[ -n "$CUSTOM_MLX_WHEEL" ]] && echo 1) FILTER_MLX_LM=$([[ -n "$CUSTOM_MLX_LM_WHEEL" ]] && echo 1) requirements.txt > "$REQ_TMP"
python -m pip install ${PIP_ARGS:-} -r "$REQ_TMP"
rm -f "$REQ_TMP"
else
python -m pip install ${PIP_ARGS:-} -r requirements.txt
fi
fi
if [[ -n "${MLX_REPO_REF:-}" ]]; then
echo "[opt] Overriding MLX from: ${MLX_REPO_REF}"
python -m pip install ${PIP_ARGS:-} --no-deps "${MLX_REPO_REF}"
fi
if [[ -n "${MLX_LM_REPO_REF:-}" ]]; then
echo "[opt] Overriding MLX-LM from: ${MLX_LM_REPO_REF}"
python -m pip install ${PIP_ARGS:-} --no-deps "${MLX_LM_REPO_REF}"
fi
python -m pip install ${PIP_ARGS:-} .
if [[ "${USE_HF_XET:-1}" == "1" ]]; then
echo "[opt] Installing Hugging Face Xet plugin..."
python -m pip install "huggingface_hub[hf_xet]" || true
fi
if [[ -n "$CUSTOM_MLX_WHEEL" ]]; then
echo "[opt] Installing custom MLX wheel: $CUSTOM_MLX_WHEEL"
python -m pip install ${PIP_ARGS:-} --force-reinstall "$CUSTOM_MLX_WHEEL"
fi
if [[ -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
echo "[opt] Installing custom MLX-LM wheel: $CUSTOM_MLX_LM_WHEEL"
python -m pip install ${PIP_ARGS:-} --force-reinstall "$CUSTOM_MLX_LM_WHEEL"
fi
python -m pip install "pyinstaller>=6.6"
# -----------------------------------------------------------------------------
# Entry script for PyInstaller
# -----------------------------------------------------------------------------
ENTRY_SCRIPT="$ROOT_DIR/.entry_mlxk_build.py"
cat > "$ENTRY_SCRIPT" <<'PY'
"""
@@ -66,10 +240,11 @@ Implements supervised shutdown by default in a frozen binary:
- Parent supervises: SIGTERM on first Ctrl-C, SIGKILL on timeout/second Ctrl-C.
Also supports a child mode keyed by MLXK2_CHILD_SERVER=1 to run the server
directly without going through CLI argument parsing.
directly without going through CLI argument parsing.
"""
import os
import json
import signal
import subprocess
import sys
@@ -81,7 +256,7 @@ def _to_bool(val: str) -> bool:
# Fast path: strip CLI name from version output for compatibility
if "--version" in sys.argv and "--json" not in sys.argv:
if "--version" in sys.argv and "--json" not in sys.argv and "--python-info" not in sys.argv:
try:
from mlxk2 import __version__ as _ver
except Exception:
@@ -89,6 +264,93 @@ if "--version" in sys.argv and "--json" not in sys.argv:
print(_ver)
sys.exit(0)
# Diagnostic: print embedded Python details
if "--python-info" in sys.argv or os.environ.get("MLXK2_PY_INFO") == "1":
info = {
"python_version": sys.version.split(" (", 1)[0],
"executable": sys.executable,
"prefix": sys.prefix,
"base_prefix": getattr(sys, "base_prefix", None),
"frozen": bool(getattr(sys, "frozen", False)),
"meipass": getattr(sys, "_MEIPASS", None),
"platform": sys.platform,
}
print(json.dumps(info, indent=2))
sys.exit(0)
# Diagnostic: package info helpers
if "--mlx-info" in sys.argv or "--pkg-info" in sys.argv:
try:
if "--mlx-info" in sys.argv:
pkg = "mlx"
else:
idx = sys.argv.index("--pkg-info")
pkg = sys.argv[idx + 1]
import importlib
mod = importlib.import_module(pkg)
version = None
try:
import importlib.metadata as md
try:
version = md.version(pkg)
except Exception:
version = None
except Exception:
pass
if version is None:
version = getattr(mod, "__version__", None)
dist_info_dir = None
try:
import importlib.metadata as md
dist = md.distribution(pkg)
files = list(dist.files or [])
for f in files:
if str(f).endswith("METADATA"):
dist_info_dir = os.fspath(dist.locate_file(f))
dist_info_dir = os.path.dirname(dist_info_dir)
break
except Exception:
dist_info_dir = None
out = {
"package": pkg,
"version": version,
"module_file": getattr(mod, "__file__", None),
"dist_info": dist_info_dir,
}
print(json.dumps(out, indent=2))
sys.exit(0)
except Exception as e:
print(json.dumps({"error": str(e)}))
sys.exit(2)
# Diagnostic: verify MLX stack imports cleanly
if "--check-mlx-stack" in sys.argv:
out = {"ok": False, "mlx": None, "mlx_lm": None, "error": None}
try:
import importlib
mlx = importlib.import_module("mlx")
out["mlx"] = {
"version": getattr(mlx, "__version__", None),
"file": getattr(mlx, "__file__", None),
}
try:
mlx_lm = importlib.import_module("mlx_lm")
out["mlx_lm"] = {
"version": getattr(mlx_lm, "__version__", None),
"file": getattr(mlx_lm, "__file__", None),
}
except Exception as e:
out["error"] = f"import mlx_lm failed: {e}"
print(json.dumps(out, indent=2))
sys.exit(2)
out["ok"] = True
print(json.dumps(out, indent=2))
sys.exit(0)
except Exception as e:
out["error"] = str(e)
print(json.dumps(out, indent=2))
sys.exit(2)
# Child mode: run the server in-process (bypasses CLI parsing)
if os.environ.get("MLXK2_CHILD_SERVER") == "1":
from mlxk2.core.server_base import run_server as _run_server
@@ -103,7 +365,6 @@ if os.environ.get("MLXK2_CHILD_SERVER") == "1":
_run_server(host=host, port=port, max_tokens=max_tokens, reload=reload, log_level=log_level)
sys.exit(0)
# Parent mode: patch start_server to supervise a child process
from mlxk2.core.server_base import run_server as _run_server
import mlxk2.operations.serve as _serve
@@ -119,22 +380,18 @@ def _run_supervised_server(*, host: str, port: int, log_level: str, reload: bool
if max_tokens is not None:
env["MLXK2_MAX_TOKENS"] = str(max_tokens)
# Start child in a new process group for clean signaling
proc = subprocess.Popen([sys.executable], start_new_session=True, env=env)
# Install signal handlers so SIGINT/SIGTERM to the parent trigger supervised cleanup
shutting_down = {"done": False}
def _graceful_shutdown(_sig=None, _frame=None):
if shutting_down["done"]:
return
shutting_down["done"] = True
# Ask child to stop gracefully
try:
os.killpg(proc.pid, signal.SIGTERM)
except Exception:
pass
# Wait briefly, then escalate to SIGKILL if needed
deadline = time.time() + 5.0
while time.time() < deadline:
ret = proc.poll()
@@ -160,15 +417,12 @@ def _run_supervised_server(*, host: str, port: int, log_level: str, reload: bool
try:
return proc.wait()
except KeyboardInterrupt:
# Suppress further SIGINT while we clean up
previous = signal.signal(signal.SIGINT, signal.SIG_IGN)
try:
# First Ctrl-C: ask child to stop gracefully
try:
os.killpg(proc.pid, signal.SIGTERM)
except Exception:
pass
# Wait briefly, then force kill if still alive
deadline = time.time() + 5.0
while time.time() < deadline:
ret = proc.poll()
@@ -177,20 +431,17 @@ def _run_supervised_server(*, host: str, port: int, log_level: str, reload: bool
try:
time.sleep(0.1)
except KeyboardInterrupt:
# Second Ctrl-C: escalate to SIGKILL immediately
break
try:
os.killpg(proc.pid, signal.SIGKILL)
except Exception:
pass
# Wait for child without being interrupted
while True:
ret = proc.poll()
if ret is not None:
return ret
time.sleep(0.05)
finally:
# Restore previous handler
try:
signal.signal(signal.SIGINT, previous)
except Exception:
@@ -199,19 +450,19 @@ def _run_supervised_server(*, host: str, port: int, log_level: str, reload: bool
def _patched_start_server(*, model=None, port=8000, host="127.0.0.1", max_tokens=None,
reload=False, log_level="info", verbose=False, supervise=True):
# Always run in-process (single PID for Electron to manage)
return _run_server(host=host, port=port, max_tokens=max_tokens, reload=reload, log_level=log_level)
# Install patch before CLI main runs so cli imports get the patched version
_serve.start_server = _patched_start_server
if __name__ == "__main__":
from mlxk2.cli import main
main()
PY
# -----------------------------------------------------------------------------
# Build with PyInstaller
# -----------------------------------------------------------------------------
echo "[3/3] Building and packaging..."
PYI_OPTS=(
--clean
@@ -236,7 +487,9 @@ fi
pyinstaller "${PYI_OPTS[@]}" "$ENTRY_SCRIPT"
# -----------------------------------------------------------------------------
# Package archive
# -----------------------------------------------------------------------------
TAR_BASE="$APP_NAME"
if [[ -n "${VERSION:-}" ]]; then
TAR_BASE="$TAR_BASE-${VERSION}"
scripts/build_external_python.sh
+269 -44
View File
@@ -5,34 +5,114 @@ set -euo pipefail
# (e.g., the interpreter embedded in your Electron app at
# MyApp.app/Contents/Resources/python/bin/python3).
#
# This script vendors all Python deps and this project into a folder and
# generates a small launcher that executes: <external-python> -m mlxk2.cli ...
# with PYTHONPATH pointing at the vendored deps.
#
# Usage:
# # Preferred: point to your embedded Python
# EMBEDDED_PYTHON="/path/to/MyApp.app/Contents/Resources/python/bin/python3" \
# scripts/build_external_python.sh
#
# # Dev fallback (uses repo-local python/bin if present, else system python3)
# scripts/build_external_python.sh
#
# Optional env:
# BIN_NAME=my-cli -> output folder/launcher name (default msty-mlx-studio)
# DIST_DIR=dist-ext-python -> output root (default dist-ext-python)
# PIP_ARGS="..." -> extra pip args (e.g., --no-index --find-links ...)
# CLEAN=1 -> delete output dir before build
#
# The script vendors dependencies into dist-ext-python/<BIN_NAME>/_vendor and
# generates a launcher that executes: <python> -m mlxk2.cli ... with PYTHONPATH
# pointed at that vendor directory.
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"
# -----------------------------------------------------------------------------
# Temporary version override (VERSION env)
# -----------------------------------------------------------------------------
ORIG_VERSION="$(python3 - <<'PY'
import pathlib, re
text = pathlib.Path("mlxk2/__init__.py").read_text()
m = re.search(r'__version__\s*=\s*"([^"]+)"', text)
print(m.group(1) if m else "0.0.0")
PY
)"
TARGET_VERSION="${VERSION:-$ORIG_VERSION}"
TMP_INIT=""
TMP_README=""
RESTORE_VERSION=0
cleanup_version() {
if [[ "$RESTORE_VERSION" == "1" ]]; then
if [[ -n "$TMP_INIT" && -f "$TMP_INIT" ]]; then
cp "$TMP_INIT" mlxk2/__init__.py
rm -f "$TMP_INIT"
fi
if [[ -n "$TMP_README" && -f "$TMP_README" ]]; then
cp "$TMP_README" README.md
rm -f "$TMP_README"
fi
fi
}
trap cleanup_version EXIT
if [[ "$TARGET_VERSION" != "$ORIG_VERSION" ]]; then
echo "[version] Updating project version: $ORIG_VERSION -> $TARGET_VERSION"
TMP_INIT="$(mktemp)"
cp mlxk2/__init__.py "$TMP_INIT"
if [[ -f README.md ]]; then
TMP_README="$(mktemp)"
cp README.md "$TMP_README"
fi
RESTORE_VERSION=1
python3 - "$ORIG_VERSION" "$TARGET_VERSION" <<'PY'
import sys
import pathlib
import re
old, new = sys.argv[1:3]
def tag(ver: str) -> str:
if 'b' in ver:
base, beta = ver.split('b', 1)
if beta and beta.isdigit():
return f"v{base}-beta.{beta}"
return f"v{ver}"
init_path = pathlib.Path("mlxk2/__init__.py")
text = init_path.read_text()
text = re.sub(r'__version__\s*=\s*".*?"', f'__version__ = "{new}"', text)
init_path.write_text(text)
readme_path = pathlib.Path("README.md")
if readme_path.exists():
readme = readme_path.read_text()
readme = readme.replace(old, new)
readme = readme.replace(tag(old), tag(new))
readme_path.write_text(readme)
PY
fi
# -----------------------------------------------------------------------------
# CLI flags (custom wheels)
# -----------------------------------------------------------------------------
MLX_WHEEL_ARG=""
MLX_LM_WHEEL_ARG=""
while [[ $# -gt 0 ]]; do
case "$1" in
--mlx-wheel)
shift
MLX_WHEEL_ARG="${1:-}"
shift || true
;;
--mlx-lm-wheel)
shift
MLX_LM_WHEEL_ARG="${1:-}"
shift || true
;;
*)
echo "Unknown flag: $1" >&2
exit 1
;;
esac
done
# -----------------------------------------------------------------------------
# Output layout
# -----------------------------------------------------------------------------
APP_NAME="${BIN_NAME:-${APP_NAME:-msty-mlx-studio}}"
OUT_ROOT="${DIST_DIR:-dist-ext-python}"
OUT_DIR="$OUT_ROOT/$APP_NAME"
VENDOR_DIR="$OUT_DIR/_vendor"
# Resolve external Python
# -----------------------------------------------------------------------------
# Resolve Python interpreter used for installs
# -----------------------------------------------------------------------------
if [[ -n "${EMBEDDED_PYTHON:-}" ]]; then
PY="$EMBEDDED_PYTHON"
echo "Using EMBEDDED_PYTHON: $PY"
@@ -49,12 +129,44 @@ if [[ ! -x "$PY" ]]; then
exit 1
fi
# Clean output if requested
# -----------------------------------------------------------------------------
# Prepare output
# -----------------------------------------------------------------------------
if [[ "${CLEAN:-0}" == "1" ]]; then
rm -rf "$OUT_DIR"
fi
mkdir -p "$VENDOR_DIR"
# -----------------------------------------------------------------------------
# Custom wheel resolution
# -----------------------------------------------------------------------------
CUSTOM_MLX_WHEEL="${MLX_WHEEL:-}"
if [[ -z "$CUSTOM_MLX_WHEEL" && -n "$MLX_WHEEL_ARG" ]]; then
CUSTOM_MLX_WHEEL="$MLX_WHEEL_ARG"
fi
if [[ -z "$CUSTOM_MLX_WHEEL" && -f "$ROOT_DIR/scripts/mlx-custom.whl" ]]; then
CUSTOM_MLX_WHEEL="$ROOT_DIR/scripts/mlx-custom.whl"
fi
if [[ -n "$CUSTOM_MLX_WHEEL" ]]; then
echo "[opt] Using custom MLX wheel: $CUSTOM_MLX_WHEEL"
[[ -f "$CUSTOM_MLX_WHEEL" ]] || { echo "ERROR: MLX wheel not found: $CUSTOM_MLX_WHEEL" >&2; exit 1; }
fi
CUSTOM_MLX_LM_WHEEL="${MLX_LM_WHEEL:-}"
if [[ -z "$CUSTOM_MLX_LM_WHEEL" && -n "$MLX_LM_WHEEL_ARG" ]]; then
CUSTOM_MLX_LM_WHEEL="$MLX_LM_WHEEL_ARG"
fi
if [[ -z "$CUSTOM_MLX_LM_WHEEL" && -f "$ROOT_DIR/scripts/mlx-lm-custom.whl" ]]; then
CUSTOM_MLX_LM_WHEEL="$ROOT_DIR/scripts/mlx-lm-custom.whl"
fi
if [[ -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
echo "[opt] Using custom MLX-LM wheel: $CUSTOM_MLX_LM_WHEEL"
[[ -f "$CUSTOM_MLX_LM_WHEEL" ]] || { echo "ERROR: MLX-LM wheel not found: $CUSTOM_MLX_LM_WHEEL" >&2; exit 1; }
fi
# -----------------------------------------------------------------------------
# Vendoring strategy
# -----------------------------------------------------------------------------
if [[ "${VENDOR_VIA_WHEELS:-0}" == "1" ]]; then
echo "[1/3] Vendoring via wheels (does not execute embedded Python)"
SYS_PY="$(command -v python3)"
@@ -62,38 +174,56 @@ if [[ "${VENDOR_VIA_WHEELS:-0}" == "1" ]]; then
echo "ERROR: python3 not found on PATH for wheel download" >&2
exit 1
fi
: "${TARGET_PY_VERSION:?Set TARGET_PY_VERSION (e.g., 3.10)}"
: "${TARGET_PY_VERSION:?Set TARGET_PY_VERSION (e.g., 3.12)}"
TARGET_PLATFORM="${TARGET_PLATFORM:-macosx_11_0_arm64}"
# Derive ABI from version (e.g., 3.10 -> cp310)
ver_nodot="${TARGET_PY_VERSION/.}" # 3.10 -> 310
ver_nodot="${TARGET_PY_VERSION/.}" # 3.12 -> 312
TARGET_ABI="${TARGET_ABI:-cp${ver_nodot}}"
WHEELS_DIR="$OUT_DIR/_wheels"
rm -rf "$WHEELS_DIR" && mkdir -p "$WHEELS_DIR"
rm -rf "$WHEELS_DIR"
mkdir -p "$WHEELS_DIR"
echo "[1a] Downloading wheels for target Python ${TARGET_PY_VERSION} (${TARGET_ABI}) platform ${TARGET_PLATFORM}..."
if [[ -f requirements.txt ]]; then
"$SYS_PY" -m pip download -r requirements.txt -d "$WHEELS_DIR" \
echo "[1a] Downloading wheels for Python ${TARGET_PY_VERSION} (${TARGET_ABI}) platform ${TARGET_PLATFORM}..."
REQ_FILE="requirements.txt"
if [[ ( -n "$CUSTOM_MLX_WHEEL" || -n "$CUSTOM_MLX_LM_WHEEL" ) && -f requirements.txt ]]; then
REQ_TMP="$(mktemp)"
awk 'BEGIN{IGNORECASE=0} {
line=$0; trimmed=line; gsub(/^[ \t]+/,"",trimmed);
if (trimmed ~ /^#/ || trimmed ~ /^$/) { print line; next }
if (ENVIRON["FILTER_MLX"] && trimmed ~ /^mlx([ \t]|[><=]|$)/) { next }
if (ENVIRON["FILTER_MLX_LM"] && trimmed ~ /^mlx-lm([ \t]|[><=]|$)/) { next }
print line
}' FILTER_MLX=$([[ -n "$CUSTOM_MLX_WHEEL" ]] && echo 1) FILTER_MLX_LM=$([[ -n "$CUSTOM_MLX_LM_WHEEL" ]] && echo 1) requirements.txt > "$REQ_TMP"
REQ_FILE="$REQ_TMP"
fi
if [[ -f "$REQ_FILE" ]]; then
"$SYS_PY" -m pip download -r "$REQ_FILE" -d "$WHEELS_DIR" \
--only-binary=:all: --implementation cp --platform "$TARGET_PLATFORM" \
--python-version "${TARGET_PY_VERSION}" --abi "$TARGET_ABI" ${PIP_ARGS:-}
fi
if [[ "${REQ_FILE}" != "requirements.txt" ]]; then
rm -f "$REQ_FILE"
fi
[[ -z "$CUSTOM_MLX_WHEEL" ]] || cp "$CUSTOM_MLX_WHEEL" "$WHEELS_DIR/"
[[ -z "$CUSTOM_MLX_LM_WHEEL" ]] || cp "$CUSTOM_MLX_LM_WHEEL" "$WHEELS_DIR/"
echo "[1b] Building wheel for local package..."
"$SYS_PY" -m pip wheel -w "$WHEELS_DIR" . ${PIP_ARGS:-}
echo "[2/3] Unpacking wheels into vendor..."
shopt -s nullglob
for whl in "$WHEELS_DIR"/*.whl; do
[ -e "$whl" ] || { echo "No wheels found in $WHEELS_DIR" >&2; exit 1; }
unzip -oq "$whl" -d "$VENDOR_DIR"
done
shopt -u nullglob
# Sanity: detect missing MLX wheel for the chosen Python version
if grep -Eqi '^\s*mlx([>=<]|\s|$)' "$ROOT_DIR/requirements.txt" 2>/dev/null; then
if ! ls "$WHEELS_DIR"/mlx-*.whl >/dev/null 2>&1; then
echo "" >&2
echo >&2
echo "ERROR: No mlx wheel was downloaded for TARGET_PY_VERSION=${TARGET_PY_VERSION}." >&2
echo "MLX publishes wheels only for certain Python versions (typically 3.11/3.12)." >&2
echo "Try one of these options:" >&2
echo " - Use TARGET_PY_VERSION=3.12 (or 3.11) with VENDOR_VIA_WHEELS=1" >&2
echo " - Or omit VENDOR_VIA_WHEELS to let the embedded Python build from source (requires unquarantined/signed interpreter)" >&2
echo "MLX publishes wheels only for select Python versions (typically 3.11/3.12)." >&2
echo "Use TARGET_PY_VERSION=3.12 (or 3.11) or disable VENDOR_VIA_WHEELS to build from source." >&2
exit 1
fi
fi
@@ -110,27 +240,43 @@ PY
echo "[2/3] Installing project dependencies into: $VENDOR_DIR"
if [[ -f requirements.txt ]]; then
"$PY" -m pip install --target "$VENDOR_DIR" -r requirements.txt ${PIP_ARGS:-}
if [[ -n "$CUSTOM_MLX_WHEEL" || -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
REQ_TMP="$(mktemp)"
awk 'BEGIN{IGNORECASE=0} {
line=$0; trimmed=line; gsub(/^[ \t]+/,"",trimmed);
if (trimmed ~ /^#/ || trimmed ~ /^$/) { print line; next }
if (ENVIRON["FILTER_MLX"] && trimmed ~ /^mlx([ \t]|[><=]|$)/) { next }
if (ENVIRON["FILTER_MLX_LM"] && trimmed ~ /^mlx-lm([ \t]|[><=]|$)/) { next }
print line
}' FILTER_MLX=$([[ -n "$CUSTOM_MLX_WHEEL" ]] && echo 1) FILTER_MLX_LM=$([[ -n "$CUSTOM_MLX_LM_WHEEL" ]] && echo 1) requirements.txt > "$REQ_TMP"
"$PY" -m pip install --target "$VENDOR_DIR" -r "$REQ_TMP" ${PIP_ARGS:-}
rm -f "$REQ_TMP"
else
"$PY" -m pip install --target "$VENDOR_DIR" -r requirements.txt ${PIP_ARGS:-}
fi
fi
echo "[2/3b] Installing mlxk2 package into vendor..."
"$PY" -m pip install --target "$VENDOR_DIR" . ${PIP_ARGS:-}
if [[ -n "$CUSTOM_MLX_WHEEL" ]]; then
echo "[opt] Installing custom MLX wheel into vendor..."
"$PY" -m pip install --target "$VENDOR_DIR" --force-reinstall "$CUSTOM_MLX_WHEEL" ${PIP_ARGS:-}
fi
if [[ -n "$CUSTOM_MLX_LM_WHEEL" ]]; then
echo "[opt] Installing custom MLX-LM wheel into vendor..."
"$PY" -m pip install --target "$VENDOR_DIR" --force-reinstall "$CUSTOM_MLX_LM_WHEEL" ${PIP_ARGS:-}
fi
fi
# -----------------------------------------------------------------------------
# Launcher
# -----------------------------------------------------------------------------
echo "[3/3] Writing launcher: $OUT_DIR/$APP_NAME"
cat > "$OUT_DIR/$APP_NAME" <<'SH'
#!/usr/bin/env bash
set -euo pipefail
# Launcher for running mlxk2 against an external Python interpreter.
#
# Picks the interpreter from:
# 1) MLXK_PYTHON override
# 2) MyApp Electron resources path (macOS):
# "$RESOURCES_PATH/python/bin/python3" if RESOURCES_PATH is set
# 3) repo-local ./python/bin/python3 relative to this launcher
# 4) system python3
SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
VENDOR_DIR="$SELF_DIR/_vendor"
@@ -160,6 +306,84 @@ PY
exit 0
fi
if [[ "${1:-}" == "--mlx-info" || "${1:-}" == "--pkg-info" ]]; then
cmd="$1"; shift || true
pkg="mlx"
if [[ "$cmd" == "--pkg-info" ]]; then
pkg="${1:-mlx}"; shift || true
fi
"$PY" - <<PY
import json, importlib, sys, os
pkg = ${pkg@P}
try:
mod = importlib.import_module(pkg)
version = None
try:
import importlib.metadata as md
try:
version = md.version(pkg)
except Exception:
version = None
except Exception:
pass
if version is None:
version = getattr(mod, "__version__", None)
dist_info_dir = None
try:
import importlib.metadata as md
dist = md.distribution(pkg)
files = list(dist.files or [])
for f in files:
if str(f).endswith("METADATA"):
dist_info_dir = os.fspath(dist.locate_file(f))
dist_info_dir = os.path.dirname(dist_info_dir)
break
except Exception:
dist_info_dir = None
print(json.dumps({
"package": pkg,
"version": version,
"module_file": getattr(mod, "__file__", None),
"dist_info": dist_info_dir,
}, indent=2))
except Exception as e:
print(json.dumps({"error": str(e)}))
sys.exit(2)
PY
exit 0
fi
if [[ "${1:-}" == "--check-mlx-stack" ]]; then
"$PY" - <<'PY'
import json
out = {"ok": False, "mlx": None, "mlx_lm": None, "error": None}
try:
import importlib
mlx = importlib.import_module("mlx")
out["mlx"] = {
"version": getattr(mlx, "__version__", None),
"file": getattr(mlx, "__file__", None),
}
try:
mlx_lm = importlib.import_module("mlx_lm")
out["mlx_lm"] = {
"version": getattr(mlx_lm, "__version__", None),
"file": getattr(mlx_lm, "__file__", None),
}
except Exception as exc:
out["error"] = f"import mlx_lm failed: {exc}"
print(json.dumps(out, indent=2))
raise SystemExit(2)
out["ok"] = True
print(json.dumps(out, indent=2))
except Exception as exc:
out["error"] = str(exc)
print(json.dumps(out, indent=2))
raise SystemExit(2)
PY
exit 0
fi
export PYTHONNOUSERSITE=1
if [[ -n "${PYTHONPATH:-}" ]]; then
export PYTHONPATH="$VENDOR_DIR:$PYTHONPATH"
@@ -176,5 +400,6 @@ cp -f LICENSE "$OUT_DIR/" 2>/dev/null || true
echo "Built external-Python dist at: $OUT_DIR"
echo "Quick test:"
echo " $OUT_DIR/$APP_NAME --python-info"
echo " $OUT_DIR/$APP_NAME --check-mlx-stack"
echo " MLXK_PYTHON=\"/path/to/MyApp.app/Contents/Resources/python/bin/python3\" \\
$OUT_DIR/$APP_NAME --version"