feat(locales): community translations

- locales/en.json: 1,850 UI strings, generated from source; translate by copying it to <tag>.json
- locales/README.md: how to add a language, what the checker refuses and why
- scripts/validate-locales.ts: the gate; rejects escape/control chars, bidi overrides, invisible chars, broken placeholders and plurals, oversize files
- .github/workflows/locales.yml: runs the gate on every PR, refuses edits to en.json, actions pinned by SHA
- CODEOWNERS: gate and en.json need maintainer review
This commit is contained in:
Pouiiro committed 2026-08-23 17:56:25 +02:00
1 parent eff0c8de75
commit 9f4f839e13
6 files changed
+2466

No files matched your search

+9
View File
@@ -22,6 +22,15 @@
/.github/workflows/ @proxysoul
/package.json @proxysoul
# Locale gate — a security control. Translations (`/locales/<tag>.json`) are
# welcome from anyone; the gate that checks them and the generated `en.json`
# are not editable by contribution, and a PR that touches them alongside a
# translation should be read as a question, not a translation.
/scripts/validate-locales.ts @proxysoul
/scripts/validate-locales.test.ts @proxysoul
/locales/en.json @proxysoul
/locales/README.md @proxysoul
# License / governance docs.
/LICENSE @proxysoul
/COMMERCIAL_LICENSE.md @proxysoul
+68
View File
@@ -0,0 +1,68 @@
# Locale gate.
#
# Translations arrive here as pull requests from people we have never met, and
# the text they carry is painted into a terminal on somebody else's machine.
# This check runs on every pull request that touches a locale file.
#
# What this workflow is, honestly: friction for honest mistakes and a fast
# signal for contributors. It is NOT the security boundary. A pull request from
# a fork runs the workflow AS MODIFIED BY THAT PULL REQUEST, so a hostile change
# can edit this file to `run: true` and show green. The control that actually
# holds is that every merged locale is re-validated by the maintainer, locally,
# with a copy of the gate this repository cannot touch, before it is embedded
# in any build. Maintainers: a green check here is a reason to look, not a
# reason to skip looking — read the "Files changed" tab, and if a translation
# PR touches anything outside `locales/<tag>.json`, ask why.
name: locales
# The pull_request trigger is NOT path-filtered, on purpose. This check is a
# required status on `main`, and a required check that only runs for some PRs
# blocks every other PR forever ("expected — waiting for status"). The gate
# reads a few JSON files with no install step, so running it on every PR costs
# seconds and lets it be required safely.
on:
pull_request:
push:
paths:
- "locales/**"
permissions:
contents: read
jobs:
validate:
runs-on: ubuntu-latest
steps:
# Actions pinned to commit SHAs, not tags. A tag can be moved; a moved
# tag on a setup action would let someone swap the `bun` that runs the
# gate for one that always exits 0.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: "1.3.13"
# `en.json` is generated from the application source and published here.
# It is not something a translation changes. A PR that edits it is either
# an honest mistake (someone "fixing" the English) or an attempt to add a
# key so their own file passes the unknown-key check — and in both cases
# the answer is the same.
- name: en.json is generated and must not change in a PR
if: github.event_name == 'pull_request'
run: |
# One shallow fetch of the base tip is all the diff needs — no full
# history clone for a check that reads one file.
git fetch --depth=1 origin "${{ github.base_ref }}"
if ! git diff --quiet FETCH_HEAD -- locales/en.json; then
echo "::error file=locales/en.json::en.json is generated from source and must not be edited here. Open an issue if a source string is wrong."
exit 1
fi
# No install step, deliberately. The gate reads JSON and uses nothing but
# the standard library, so a translation pull request can never cause a
# postinstall script to run on CI.
- name: Validate locale files
run: bun scripts/validate-locales.ts
- name: Run the gate's own tests
run: bun test scripts/validate-locales.test.ts
+126
View File
@@ -0,0 +1,126 @@
# Translating Empryo
Every word Empryo shows you lives in this folder. `en.json` is the source, written by
the code. Every other file is a translation of it.
You do not need to know TypeScript to add a language. You need a text editor and a
language you speak well.
## Add a language
1. Copy `en.json` to your language's code. Use the
[BCP-47 tag](https://en.wikipedia.org/wiki/IETF_language_tag): `ja.json` for Japanese,
`ar.json` for Arabic, `zh-CN.json` for Simplified Chinese, `pt-BR.json` for Brazilian
Portuguese.
2. Translate the values. Leave the keys exactly as they are.
3. Run `bun scripts/validate-locales.ts` and fix anything it reports.
4. Open a pull request.
```json
{
"common.action.cancel": "キャンセル",
"common.label.files": "ファイル"
}
```
**You do not have to finish.** Translate fifty keys and stop. Anything you leave out
falls back to English, key by key, so a partial translation ships and works. Someone
else — maybe you, later — picks up the rest.
## Rules
**Keep the keys.** The left side of every pair is an address the code looks things up
by. Change it and the text disappears.
**Keep the `{placeholders}`.** `{name}`, `{count}` and friends get replaced with real
values at runtime. Every one in the English string must appear in yours, spelled the
same. You can move them anywhere in the sentence — that is the point of them.
```json
"session.greeting": "Welcome back, {name}" ← English
"session.greeting": "{name}さん、おかえりなさい" ← fine, moved to the front
"session.greeting": "おかえりなさい" ← rejected, {name} is gone
```
**Counting words need `plural`.** English has two forms; your language may have one, or
six. Write the ones your language actually uses, and always include `other`:
```json
"session.steps": "{count, plural, other {# 步}}",
"session.steps": "{count, plural, one {# krok} few {# kroki} many {# kroków} other {# kroku}}",
"session.steps": "{count, plural, zero {لا خطوات} one {خطوة} two {خطوتان} few {# خطوات} other {# خطوة}}"
```
`#` becomes the number, written the way your language writes numbers.
**Do not translate product names.** Empryo, Claude, OpenAI, GitHub, JSON, API. They stay
as they are.
**Do not paste from a terminal.** Copying text out of a terminal can bring invisible
escape codes with it. The checker rejects those, and it is right to — see below.
**Keep it short.** Much of this text sits in a fixed-width panel. If English says
"Compact", a six-word translation will not fit. Aim for roughly the English length.
## What the checker refuses, and why
Run `bun scripts/validate-locales.ts` before you open a pull request. It fails on:
| It rejects | Because |
|---|---|
| Escape and control characters | A terminal does not *print* these, it *obeys* them. One particular sequence writes to your clipboard. Another repaints the screen, which is enough to fake a confirmation prompt. None of it shows up in a diff. |
| Bidi override characters | They let the text a reviewer reads differ from the text a user sees. Arabic and Hebrew do not need them — the direction comes from the letters themselves. |
| Zero-width and invisible characters | Same problem: things nobody can see in review. |
| A missing or invented `{placeholder}` | The message either loses information or crashes when it is formatted. |
| A plural that lost its `plural` wrapper, or its `other` branch | It would print the word "count" instead of a number, in the one language nobody reviewing can read. |
| Values over 400 characters | At that length it is a payload, not a label. |
| Keys that are not in `en.json` | Stale after a rename, or invented. Either way the code will never read them. |
This is strict because these files come from strangers on the internet and end up
painted into a terminal on someone else's machine. Nothing personal.
## Right-to-left languages
Arabic, Hebrew, Persian and Urdu are welcome. Be aware of where they land:
- **The desktop app** handles right-to-left properly. The layout mirrors.
- **The terminal interface** does not, and cannot — terminals do not reorder
bidirectional text. Your translation will be correct but the layout around it will
still read left-to-right.
Translate anyway. The desktop app is the better home for it, and a terminal in your
language beats a terminal in mine.
## Chinese, Japanese, Korean
These characters take **two columns** in a terminal where a Latin letter takes one. The
code accounts for it, so you do not have to. Just keep translations near the English
length in *visual* width — four ideographs occupy about as much room as eight letters.
## When English changes
New keys appear in `en.json` when features are added. Your file does not need updating
immediately — the new keys just show in English until someone translates them. Run the
checker any time to see your coverage:
```
ok ja 68% translated (848/1247)
```
## Where `en.json` comes from
It is generated, not written by hand. The strings are extracted from the application
source and published here so you have something to translate against. That means:
- **Do not edit `en.json`.** Your changes would be overwritten the next time it is
published. If a source string itself is wrong or unclear, open an issue instead.
- New keys appear when features are added. Your file does not need to keep up — the
new keys just show in English until somebody translates them.
## What happens to your pull request
Once merged, it is copied into the build tree, checked again, and committed there. From
that point it ships in the next release. The commit your file came from is recorded, so
the text in any released binary can be traced back to the pull request that wrote it.
You do not need to do anything for that to happen.
+1852
View File
File diff suppressed because it is too large. Load diff
+187
View File
@@ -0,0 +1,187 @@
/**
* The locale gate is a security control, and a security control that has never
* been shown a hostile input is a wish. Each case below is a real thing a
* translation pull request could contain, built from escape sequences so this
* file stays readable in a diff.
*/
import { describe, expect, test } from "bun:test";
import { mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const SCRIPT = join(import.meta.dir, "validate-locales.ts");
/** English source used by every case. Small, with one plural and one placeholder. */
const EN = {
"common.action.cancel": "Cancel",
"common.label.files": "Files",
"session.steps": "{count, plural, one {# step} other {# steps}}",
"session.greeting": "Welcome back, {name}",
"common.state.ready": "Ready",
};
/** Run the gate over a throwaway locale tree. */
function gate(locale: Record<string, unknown>): { code: number; out: string } {
const dir = mkdtempSync(join(tmpdir(), "empryo-locales-"));
writeFileSync(join(dir, "en.json"), JSON.stringify(EN));
writeFileSync(join(dir, "zz.json"), JSON.stringify(locale));
const proc = Bun.spawnSync(["bun", SCRIPT], {
env: { ...process.env, LOCALES_DIR: dir },
});
return {
code: proc.exitCode ?? -1,
out: proc.stdout.toString() + proc.stderr.toString(),
};
}
describe("locale gate", () => {
test("accepts an honest translation", () => {
const { code, out } = gate({
"common.action.cancel": "取消",
"common.label.files": "文件",
"session.steps": "{count, plural, other {# 步}}",
"session.greeting": "欢迎回来,{name}",
});
expect(code).toBe(0);
expect(out).toContain("ok");
});
test("accepts right-to-left text without explicit overrides", () => {
// Arabic derives direction from its own letters; no control marks needed.
const { code } = gate({ "common.action.cancel": "إلغاء", "common.label.files": "الملفات" });
expect(code).toBe(0);
});
test("refuses OSC 52, which would write the user's clipboard", () => {
const osc = `\u001b]52;c;${btoa("stolen")}\u0007Cancel`;
const { code, out } = gate({ "common.action.cancel": osc });
expect(code).not.toBe(0);
expect(out).toContain("control-char");
});
test("refuses cursor-movement sequences that could forge a prompt", () => {
const { code, out } = gate({ "common.action.cancel": "\u001b[2J\u001b[HCancel" });
expect(code).not.toBe(0);
expect(out).toContain("control-char");
});
test("refuses C1 controls, which some terminals treat as escape introducers", () => {
const { code, out } = gate({ "common.action.cancel": "Cancel\u009b0m" });
expect(code).not.toBe(0);
expect(out).toContain("control-char");
});
test("refuses bidi overrides (Trojan Source)", () => {
const { code, out } = gate({ "common.action.cancel": "\u202eCancel" });
expect(code).not.toBe(0);
expect(out).toContain("bidi-override");
});
test("refuses invisible padding", () => {
const { code, out } = gate({ "common.action.cancel": `Cancel${"\u200b".repeat(20)}` });
expect(code).not.toBe(0);
expect(out).toContain("invisible");
});
test("refuses an oversized value", () => {
const { code, out } = gate({ "common.action.cancel": "x".repeat(500) });
expect(code).not.toBe(0);
expect(out).toContain("length");
});
test("refuses a dropped placeholder", () => {
const { code, out } = gate({ "session.greeting": "Welcome back" });
expect(code).not.toBe(0);
expect(out).toContain("missing {name}");
});
test("refuses an invented placeholder", () => {
const { code, out } = gate({ "session.greeting": "Welcome, {name} ({email})" });
expect(code).not.toBe(0);
expect(out).toContain("unexpected {email}");
});
test("refuses a plural that lost its plural form", () => {
const { code, out } = gate({ "session.steps": "{count} steps" });
expect(code).not.toBe(0);
expect(out).toContain("plural");
});
test("refuses a plural with no `other` branch", () => {
const { code, out } = gate({ "session.steps": "{count, plural, one {# krok}}" });
expect(code).not.toBe(0);
expect(out).toContain("other");
});
test("refuses unbalanced braces", () => {
const { code, out } = gate({ "common.action.cancel": "Cancel {" });
expect(code).not.toBe(0);
expect(out).toContain("braces");
});
test("refuses a key that is not in the source catalog", () => {
const { code, out } = gate({ "totally.made.up": "hi" });
expect(code).not.toBe(0);
expect(out).toContain("unknown-key");
});
test("refuses a non-string value", () => {
const { code, out } = gate({ "common.action.cancel": 42 });
expect(code).not.toBe(0);
expect(out).toContain("type");
});
test("reports coverage so a partial translation reads as partial", () => {
const { out } = gate({ "common.action.cancel": "取消" });
expect(out).toMatch(/20% translated/);
});
/**
* Invisible-character classes the first version of this gate let straight
* through. Each renders as nothing, so each is a channel that survives review.
*/
test("refuses Unicode Tag characters, the ASCII-smuggling block", () => {
const smuggled = `Cancel${String.fromCodePoint(0xe0001, 0xe0041, 0xe0042)}`;
const { code, out } = gate({ "common.action.cancel": smuggled });
expect(code).not.toBe(0);
expect(out).toContain("invisible");
});
test("refuses variation selectors used as a hidden channel", () => {
for (const cp of [0xfe00, 0xe0100]) {
const { code } = gate({ "common.action.cancel": `Cancel${String.fromCodePoint(cp)}` });
expect(code).not.toBe(0);
}
});
test("refuses line and paragraph separators", () => {
for (const cp of [0x2028, 0x2029]) {
const { code } = gate({ "common.action.cancel": `Cancel${String.fromCodePoint(cp)}` });
expect(code).not.toBe(0);
}
});
test("refuses soft hyphen, ALM, MVS and interlinear annotation", () => {
for (const cp of [0x00ad, 0x061c, 0x180e, 0xfff9]) {
const { code } = gate({ "common.action.cancel": `Cancel${String.fromCodePoint(cp)}` });
expect(code).not.toBe(0);
}
});
test("still accepts ordinary text after all that tightening", () => {
const { code } = gate({
"common.action.cancel": "キャンセル",
"common.label.files": "ファイル",
"session.greeting": "{name}さん、おかえりなさい",
});
expect(code).toBe(0);
});
test("refuses a file too large to be a translation, before parsing it", () => {
// 5 MB of a valid-looking value: the size check must fire, not JSON.parse.
const { code, out } = gate({ "common.action.cancel": "x".repeat(5 * 1024 * 1024) });
expect(code).not.toBe(0);
expect(out).toContain("size");
expect(out).not.toContain("length"); // never got as far as the per-value rule
});
});
+224
View File
@@ -0,0 +1,224 @@
#!/usr/bin/env bun
/**
* Locale gate — the only thing standing between a stranger's pull request and
* the text this product paints into a terminal.
*
* Translations arrive from the public tracker. That makes every value in every
* `locales/*.json` attacker-controlled input, and the surface it reaches is not
* a sandbox: it is a TTY that interprets what it is handed. Each rule below
* exists because of something a string can DO, not because of style.
*
* bun scripts/validate-locales.ts check every locale
* bun scripts/validate-locales.ts zh-CN check one
*
* Exit 0 = safe to merge. Non-zero = do not merge.
*
* Every pattern below is written with `\u`/`\x` escapes rather than the literal
* characters. A file that contains a raw ESC in order to test for raw ESC is a
* file no reviewer can read and no diff can show honestly.
*/
import { Glob } from "bun";
/**
* Directory holding the catalogs. Overridable so tests can point the gate at a
* fixture tree instead of the shipped one — a security check nobody can run
* against hostile input is a security check nobody trusts.
*/
const DIR = process.env.LOCALES_DIR ?? "locales";
const SOURCE = `${DIR}/en.json`;
/** Longest a single value may be, in characters. Beyond this a "translation" is a payload. */
const MAX_LEN = 400;
/**
* Largest a locale file may be, in bytes. `en.json` is ~90 KB at 1,850 keys; a
* full translation in a verbose script might reach twice that. Anything past a
* few megabytes is not a translation, and `JSON.parse` on a file that size
* exhausts the CI runner before a single rule has run — a "check" that can be
* knocked over by the thing it checks is not a check.
*/
const MAX_FILE_BYTES = 4 * 1024 * 1024;
type Catalog = Record<string, string>;
interface Problem {
locale: string;
key: string;
rule: string;
detail: string;
}
const problems: Problem[] = [];
const note = (locale: string, key: string, rule: string, detail: string) =>
problems.push({ locale, key, rule, detail });
/**
* Control characters and escape introducers.
*
* This is the rule that matters. A terminal does not print `ESC`, it OBEYS it.
* A locale value carrying `OSC 52` writes the user's system clipboard; `OSC 0`
* rewrites the window title; cursor-movement sequences repaint parts of the
* screen the app believes it owns, which is enough to forge a confirmation
* prompt. None of it is visible in a diff. All of it is refused.
*
* Covers C0 (\x00-\x1f), DEL (\x7f) and C1 (\x80-\x9f). Tab and newline are
* refused too: a catalog value is a phrase, and neither belongs in one.
*/
const CONTROL = /[\u0000-\u001f\u007f-\u009f]/;
/**
* Bidirectional overrides — the Trojan Source class.
*
* These reorder glyphs at render time, so what a reviewer reads and what a user
* sees can be made to differ arbitrarily. Legitimate right-to-left text (Arabic,
* Hebrew) needs NONE of them: the bidi algorithm derives direction from the
* letters themselves. A translation that contains an explicit override is either
* broken or hostile, and both answers are "reject".
*/
const BIDI = /[\u202a-\u202e\u2066-\u2069]/;
/**
* Invisible formatting — characters that occupy no space and show in no diff.
*
* The bar here is not "does it look harmful", it is "can a reviewer see it".
* Anything that renders as nothing can carry a payload through review: a
* tracking beacon, an exfiltration marker, or text addressed to a model rather
* than to a person. The Unicode Tag block (U+E0000-E007F) is the sharp one —
* it encodes arbitrary ASCII as invisible code points and is the standard
* vehicle for smuggling instructions into text an LLM will later read.
*
* `width.ts` already treats every one of these as zero-width. A character the
* renderer knows is invisible and the gate does not is exactly the seam to
* close.
*/
const INVISIBLE =
/[\u00ad\u061c\u180e\u200b-\u200f\u2028\u2029\u2060-\u2064\ufe00-\ufe0f\ufeff\ufff9-\ufffb]|[\u{e0000}-\u{e007f}]|[\u{e0100}-\u{e01ef}]/u;
/** `{name}`, `{count, plural, …}` — the argument NAMES a message depends on. */
function placeholders(pattern: string): Set<string> {
const names = new Set<string>();
for (const m of pattern.matchAll(/\{\s*([A-Za-z_][A-Za-z0-9_]*)\s*(?:,|\})/g)) names.add(m[1]!);
return names;
}
/** Arguments used with a plural form, per pattern. */
function pluralArgs(pattern: string): Set<string> {
const out = new Set<string>();
for (const m of pattern.matchAll(/\{\s*([A-Za-z_][A-Za-z0-9_]*)\s*,\s*plural\b/g)) out.add(m[1]!);
return out;
}
function balanced(s: string): boolean {
let depth = 0;
for (const ch of s) {
if (ch === "{") depth++;
else if (ch === "}" && --depth < 0) return false;
}
return depth === 0;
}
const src = (await Bun.file(SOURCE).json()) as Catalog;
const srcKeys = new Set(Object.keys(src));
const only = process.argv[2];
const targets: string[] = [];
for await (const f of new Glob("*.json").scan(DIR)) {
const tag = f.replace(/\.json$/, "");
if (tag === "en" || (only && tag !== only)) continue;
targets.push(tag);
}
targets.sort();
if (targets.length === 0) {
console.log(`No translations yet. Source ${SOURCE} has ${srcKeys.size} keys.`);
process.exit(0);
}
for (const tag of targets) {
const file = Bun.file(`${DIR}/${tag}.json`);
if (file.size > MAX_FILE_BYTES) {
note(tag, "-", "size", `${file.size} bytes, limit is ${MAX_FILE_BYTES}`);
continue;
}
let cat: Catalog;
try {
cat = (await file.json()) as Catalog;
} catch (err) {
note(tag, "-", "parse", `not valid JSON: ${err instanceof Error ? err.message : String(err)}`);
continue;
}
if (Array.isArray(cat) || typeof cat !== "object" || cat === null) {
note(tag, "-", "shape", "top level must be a flat object of key to string");
continue;
}
// A tag we cannot construct a formatter for cannot pluralise or format numbers.
try {
new Intl.PluralRules(tag);
} catch {
note(tag, "-", "tag", `"${tag}" is not a usable BCP-47 language tag`);
}
for (const [key, value] of Object.entries(cat)) {
if (typeof value !== "string") {
note(tag, key, "type", `value is ${typeof value}, must be a string`);
continue;
}
if (!srcKeys.has(key)) {
note(tag, key, "unknown-key", "not present in en.json — stale or invented");
continue;
}
if (CONTROL.test(value)) {
note(tag, key, "control-char", "contains an escape or control character");
}
if (BIDI.test(value)) {
note(tag, key, "bidi-override", "contains an explicit bidi override");
}
if (INVISIBLE.test(value)) {
note(tag, key, "invisible", "contains zero-width or invisible formatting");
}
if (value.length > MAX_LEN) {
note(tag, key, "length", `${value.length} chars, limit is ${MAX_LEN}`);
}
if (!balanced(value)) {
note(tag, key, "braces", "unbalanced { }");
}
const want = placeholders(src[key]!);
const got = placeholders(value);
for (const p of want) if (!got.has(p)) note(tag, key, "placeholder", `missing {${p}}`);
for (const p of got) if (!want.has(p)) note(tag, key, "placeholder", `unexpected {${p}}`);
// A plural argument in English must stay a plural argument: dropping the
// wrapper turns `{count, plural, …}` into the literal word "count".
for (const p of pluralArgs(src[key]!)) {
if (!pluralArgs(value).has(p)) note(tag, key, "plural", `{${p}} lost its plural form`);
}
// Every plural needs an `other` branch — it is the only category that is
// mandatory in every language, and the runtime falls back to it.
if (pluralArgs(value).size > 0 && !/\bother\s*\{/.test(value)) {
note(tag, key, "plural", "plural is missing its `other` branch");
}
}
const have = Object.keys(cat).filter((k) => srcKeys.has(k)).length;
// Round DOWN, and never round a non-empty translation to 0: "0% translated"
// next to five real translated strings reads as a failure rather than a start.
const exact = (have / srcKeys.size) * 100;
const pct = have > 0 ? Math.max(1, Math.floor(exact)) : 0;
const bad = problems.filter((p) => p.locale === tag).length;
const status = bad === 0 ? "ok " : "FAIL";
const count = bad ? ` ${bad} problem${bad === 1 ? "" : "s"}` : "";
console.log(
`${status} ${tag.padEnd(8)} ${String(pct).padStart(3)}% translated (${have}/${srcKeys.size})${count}`,
);
}
if (problems.length > 0) {
console.log("");
for (const p of problems) {
console.log(` ${p.locale} ${p.rule.padEnd(14)} ${p.key}\n ${p.detail}`);
}
console.log(`\n${problems.length} problem${problems.length === 1 ? "" : "s"}. Not safe to merge.`);
process.exit(1);
}