Host DroidDeck's package catalogs and releases here, mirrored, fetched and built by CI (#1)

The four catalogs DroidDeck reads (components.json, desktop.json, linuxfs.json, steam-seed.json)
move here from The412Banner/winlator-contents with every link rewritten to this repository, and
every file they point at now lives on a release here, produced by a workflow:

- mirror-release.yml brings a release in: each asset is checked against what the catalog
  promises for it (md5/size from the Bottles manifests, sha256 for the packages, the files
  archive's sha256 inside each recording), uploaded under the same tag, its sha256 confirmed
  against what GitHub stored, and the run named in MIRROR.md beside a SHA256SUMS. In upstream
  mode the vendor installers a pinned Bottles manifest (2026-06-23) describes with the same md5
  are fetched from the vendor's own link, 55 of the 131 Windows packages; the rest are copies.
  Releases: system-libraries-v1, component-snapshots-v1, steamdeck-desktop-r1, linuxfs-r9,
  steam-proton-arm64-25502785.
- build-dll-bundles.yml rebuilds the 39 hand-made DLL bundles from the vendors' packages
  (tools/bundles: recipes from the Bottles manifests, opened with cabextract, 7-Zip, zipfile and
  DroidDeck's own installer unpacker), each refused unless file-for-file identical to the bundle
  it replaces; 38 built, PowerShell kept as the verified copy (origin unknown; the app leaves it
  out), the VC++ 2022 twin built from the 2026 release channel since Microsoft no longer serves
  the version the hand-made one held. BUILT.md names the run and every source.
- The recorder (component-snapshots.yml + tools/snapshot) and the desktop, emulator, AppImage
  and linuxfs builders move here, so future releases are produced here.

Two catalog errors inherited from the Bottles manifests are corrected on the way: the VC++ 2012
32-bit installer entry was the 2010 installer, and the 2010 32-bit one was the original release
while its 64-bit twin was SP1 (tools/mirror/upstream-overrides.json holds the vendor links).
Nothing on any release was uploaded by hand.
This commit is contained in:
The412Banner authored and GitHub committed 2026-10-10 16:46:59 -04:00
1 parent 780ab619de
commit a9ce53ffb3
22 files changed
+8778

No files matched your search

+55
View File
@@ -0,0 +1,55 @@
name: Build desktop packages (SteamDeck)
# The hosted packages the SteamDeck app installs into its Linux runtime on request: the LXQt/labwc
# desktop and the tier-1 emulator set, each the closure of a seed list over Arch Linux ARM minus
# what the runtime image already carries. Published to one release the app's desktop.json points at.
on:
workflow_dispatch:
inputs:
release:
description: 'Release tag to publish to (created if missing)'
required: true
default: 'steamdeck-desktop-r1'
packages:
description: 'Space-separated package names (seed files under desktop/seeds/)'
required: true
default: 'desktop emulators'
permissions:
contents: write
jobs:
build:
runs-on: ubuntu-24.04
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Build
run: |
set -euo pipefail
sudo apt-get update -qq && sudo apt-get install -y -qq zstd >/dev/null
mkdir -p out
for name in ${{ inputs.packages }}; do
desktop/build-pkg.sh "$name" "desktop/seeds/$name.txt" desktop/base-packages.txt out
done
ls -l out
- name: Publish
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
TAG='${{ inputs.release }}'
if ! gh release view "$TAG" >/dev/null 2>&1; then
gh release create "$TAG" --prerelease --title "SteamDeck desktop packages ($TAG)" \
--notes "Hosted packages for the SteamDeck app's Linux runtime. Each extracts over the r9 rootfs; see desktop.json for the catalog."
fi
gh release upload "$TAG" out/* --clobber
echo "published:"; for f in out/*.tar.zst; do n=$(basename "$f" .tar.zst); echo " $n $(stat -c %s "$f") $(cat out/$n.sha256)"; done
- uses: actions/upload-artifact@v4
with:
name: desktop-packages
path: out/*.packages.txt
+79
View File
@@ -0,0 +1,79 @@
name: Build the DLL bundles
# Rebuilds the DLL bundles on system-libraries-v1 from the vendors' own packages
# (tools/bundles/recipes.json), checks each against the hand-made bundle it replaces file for
# file (tools/bundles/expect.json), packs it reproducibly and publishes it, with BUILT.md naming
# this run and every source package's origin and md5. A bundle that differs is refused.
on:
workflow_dispatch:
inputs:
bundles:
description: 'Space-separated bundle file names, or all'
required: true
default: 'all'
release:
description: 'Release the bundles are published to'
required: true
default: 'system-libraries-v1'
permissions:
contents: write
# One writer at a time on the release (the mirror workflow shares this group).
concurrency:
group: system-libraries-v1
cancel-in-progress: false
jobs:
build:
runs-on: ubuntu-24.04
timeout-minutes: 180
steps:
- uses: actions/checkout@v4
- name: Tools (cabextract, 7-Zip, msitools for DroidDeck's unpacker)
run: |
set -euo pipefail
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends cabextract p7zip-full msitools >/dev/null
# DroidDeck's own installer unpacker, as the phone runs it (pinned to the commit that merged it).
curl -fsSL -o droiddeck-msi-install "https://raw.githubusercontent.com/Droid-Deck/DroidDeck/9b90386e73b83ac5b43a64831b150c7852c18e38/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install" \
|| curl -fsSL -o droiddeck-msi-install "https://raw.githubusercontent.com/Droid-Deck/DroidDeck/main/tools/linuxfs/overlay/usr/local/bin/droiddeck-msi-install"
python3 -m py_compile droiddeck-msi-install
echo "DROIDDECK_MSIINFO=$(command -v msiinfo)" >> "$GITHUB_ENV"
echo "DROIDDECK_CABEXTRACT=$(command -v cabextract)" >> "$GITHUB_ENV"
echo "DROIDDECK_7Z=$(command -v 7z)" >> "$GITHUB_ENV"
- name: Build
env:
GH_TOKEN: ${{ github.token }}
BUNDLES: ${{ inputs.bundles || 'all' }}
run: |
set -euo pipefail
only=""
[ "$BUNDLES" = all ] || only="--only $BUNDLES"
python3 tools/bundles/build.py --recipes tools/bundles/recipes.json --expect tools/bundles/expect.json \
--out out --cache cache --unpacker ./droiddeck-msi-install $only 2>&1 | tee build.log
{ echo "## DLL bundles"; echo; tail -1 build.log; echo; grep -E '^ (built|skipped|REFUSED)' build.log | sed 's/^ /- /'; } >> "$GITHUB_STEP_SUMMARY"
- name: Publish
if: success()
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.release || 'system-libraries-v1' }}
run: |
set -euo pipefail
sed -i "1a\\\\nBuilt by $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID.\n" out/BUILT.md
gh release upload "$TAG" out/* --clobber
# SHA256SUMS from what GitHub now holds, so it covers the rebuilt files.
gh api "repos/$GITHUB_REPOSITORY/releases/tags/$TAG" --paginate --jq '.assets[] | select(.name!="SHA256SUMS" and .name!="MIRROR.md" and .name!="BUILT.md") | "\(.digest|sub("^sha256:";"")) \(.name)"' | sort -k2 > SHA256SUMS
gh release upload "$TAG" SHA256SUMS --clobber
- uses: actions/upload-artifact@v4
if: always()
with:
name: dll-bundles
path: |
out/
build.log
+117
View File
@@ -0,0 +1,117 @@
name: Build emulator from source (SteamDeck, tier 3)
# The emulators that ship no ARM64 Linux binary — PCSX2, Dolphin, Cemu — built on GitHub's ARM
# runner and published as AppImages to the desktop release. Each is a large build; one at a time.
on:
workflow_dispatch:
inputs:
target:
description: 'dolphin | pcsx2 | cemu'
required: true
default: 'dolphin'
release:
description: 'Release tag to publish to (none = keep the AppImage as a run artifact only)'
required: true
default: 'steamdeck-desktop-r1'
cemu_ref:
description: 'Cemu commit or branch to build (cemu only; empty = main)'
required: false
default: ''
permissions:
contents: write
jobs:
build:
runs-on: ubuntu-24.04-arm
timeout-minutes: 340
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: desktop
- name: Toolchain
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends git cmake ninja-build build-essential \
pkg-config python3 curl file desktop-file-utils libfuse2 \
qt6-base-dev qt6-base-private-dev qt6-svg-dev qt6-tools-dev \
libgl1-mesa-dev libvulkan-dev libsdl2-dev libx11-dev libxext-dev libwayland-dev \
libudev-dev libevdev-dev libasound2-dev libpulse-dev libcurl4-openssl-dev \
zlib1g-dev libzstd-dev liblz4-dev libbz2-dev libpng-dev libhidapi-dev \
libbluetooth-dev libusb-1.0-0-dev libsystemd-dev gettext
- name: Fetch linuxdeploy (AppImage packager, aarch64)
run: |
set -euo pipefail
for tool in linuxdeploy linuxdeploy-plugin-qt; do
curl -fsSL -o "/usr/local/bin/$tool" \
"https://github.com/linuxdeploy/$tool/releases/download/continuous/$tool-aarch64.AppImage"
chmod +x "/usr/local/bin/$tool"
done
- name: Build ${{ inputs.target }}
id: build
run: |
set -euo pipefail
T='${{ inputs.target }}'
case "$T" in
dolphin)
git clone --depth 1 https://github.com/dolphin-emu/dolphin.git src
git -C src submodule update --init --recursive --depth 1
cmake -S src -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DLINUX_LOCAL_DEV=false \
-DDISTRIBUTOR=SteamDeck -DENABLE_TESTS=OFF -DCMAKE_INSTALL_PREFIX=/usr
cmake --build build -j"$(nproc)"
DESTDIR="$PWD/AppDir" cmake --install build
# The install already placed the entry and icon; point at those copies.
APPRUN_BIN=dolphin-emu; DESKTOP=AppDir/usr/share/applications/dolphin-emu.desktop
ICON=AppDir/usr/share/icons/hicolor/256x256/apps/dolphin-emu.png
;;
pcsx2|cemu)
# Newer libraries than the runner image has (Qt 6.10 for PCSX2, wxWidgets 3.3 for
# Cemu): built in an Arch Linux ARM container, the rootfs's own distribution.
docker run --rm -v "$PWD:/work" -w /work -e CEMU_REF='${{ inputs.cemu_ref }}' menci/archlinuxarm:base-devel \
bash desktop/build-emulator-arch.sh "$T"
sudo chown -R "$(id -u):$(id -g)" .
echo "packaged=yes" >> "$GITHUB_OUTPUT"
exit 0
;;
*) echo "unknown target"; exit 2 ;;
esac
# The desktop entry and icon appimagetool insists on, placed where linuxdeploy looks for
# them. The icon takes the name the entry's Icon= line asks for, or the two never meet.
ICONNAME=$(sed -n 's/^Icon=//p' "$DESKTOP" | head -1); ICONNAME=${ICONNAME:-$APPRUN_BIN}
D="AppDir/usr/share/applications/$(basename "$DESKTOP")"; I="AppDir/usr/share/icons/hicolor/256x256/apps/$ICONNAME.png"
[ "$DESKTOP" -ef "$D" ] || install -Dm644 "$DESKTOP" "$D"
[ "$ICON" -ef "$I" ] || install -Dm644 "$ICON" "$I"
# -print -quit, not a pipe into head: under pipefail a closed pipe would end the step.
BIN=$(find AppDir/usr -maxdepth 3 -type f -name "$APPRUN_BIN" -print -quit)
[ -n "$BIN" ] || { echo "no $APPRUN_BIN in AppDir"; find AppDir -maxdepth 3 -type d; exit 3; }
echo "apprun=$BIN" >> "$GITHUB_OUTPUT"
echo "desktop=AppDir/usr/share/applications/$(basename "$DESKTOP")" >> "$GITHUB_OUTPUT"
- name: Package AppImage
if: steps.build.outputs.packaged != 'yes'
run: |
set -euo pipefail
export OUTPUT='${{ inputs.target }}.AppImage' QMAKE=/usr/lib/qt6/bin/qmake6 EXTRA_QT_PLUGINS=svg
linuxdeploy --appdir AppDir -e '${{ steps.build.outputs.apprun }}' \
-d '${{ steps.build.outputs.desktop }}' --plugin qt --output appimage
ls -l *.AppImage
sha256sum '${{ inputs.target }}.AppImage' | awk '{print $1}' > '${{ inputs.target }}.sha256'
- uses: actions/upload-artifact@v4
if: inputs.release == 'none'
with:
name: ${{ inputs.target }}-appimage
path: |
${{ inputs.target }}.AppImage
${{ inputs.target }}.sha256
- name: Publish
if: inputs.release != 'none'
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
gh release upload '${{ inputs.release }}' '${{ inputs.target }}.AppImage' '${{ inputs.target }}.sha256' --clobber
echo "published ${{ inputs.target }} $(stat -c %s '${{ inputs.target }}.AppImage') $(cat '${{ inputs.target }}.sha256')"
+65
View File
@@ -0,0 +1,65 @@
name: Build linuxfs (DroidDeck base runtime)
# The trimmed base runtime (linuxfs/build-base.sh) as a run artifact, and optionally attached to
# a release tag of this repo. The catalog (linuxfs.json on main) is never touched here.
on:
workflow_dispatch:
inputs:
label:
description: "Label for the artifact and tar name (e.g. r10)"
required: true
default: "r10"
turnip_url:
description: "URL of the -Linux Turnip zip to ship as the default driver"
required: true
default: ""
release:
description: "Release tag to attach linuxfs.tar.zst to (empty = artifact only)"
required: false
default: ""
permissions:
contents: write
jobs:
linuxfs:
runs-on: ubuntu-24.04
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Install build dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends curl tar zstd binutils xz-utils unzip python3 proot qemu-user-static
- name: Build the rootfs
env:
TURNIP_URL: ${{ inputs.turnip_url }}
run: |
mkdir -p out
linuxfs/build-base.sh "${RUNNER_TEMP}/linuxfs-work" "${GITHUB_WORKSPACE}/out/linuxfs.tar.zst"
- name: Report
run: |
set -e
ls -l out/linuxfs.tar.zst
sha256sum out/linuxfs.tar.zst | cut -d' ' -f1 > out/linuxfs.sha256
{
echo "### linuxfs ${{ inputs.label }}"
echo ""
echo "| | |"
echo "|---|---|"
echo "| size | $(stat -c %s out/linuxfs.tar.zst) bytes |"
echo "| sha256 | \`$(cat out/linuxfs.sha256)\` |"
echo "| turnip | ${{ inputs.turnip_url }} |"
} >> "$GITHUB_STEP_SUMMARY"
- uses: actions/upload-artifact@v4
with:
name: linuxfs-${{ inputs.label }}
path: |
out/linuxfs.tar.zst
out/linuxfs.sha256
out/linuxfs-packages.txt
- name: Attach to release
if: inputs.release != ''
env:
GH_TOKEN: ${{ github.token }}
run: gh release upload '${{ inputs.release }}' out/linuxfs.tar.zst out/linuxfs.sha256 out/linuxfs-packages.txt --clobber
+111
View File
@@ -0,0 +1,111 @@
name: Record component snapshots
# Runs a Windows component's installer once, unattended, in a fresh Wine prefix on an x86 runner
# and records what it left behind: the registry values it wrote (the point: self-registering
# DLLs decide their own entries at run time) and the files it placed, each mapped back to where
# the same bytes sit inside the vendor's installer (innoextract / 7-Zip), so a device can pull them
# from that installer rather than from a copy hosted here. Recipes: tools/snapshot/recipes.json.
# Output per component: <name>.snapshot.json + <name>.summary.md on one release.
on:
workflow_dispatch:
inputs:
components:
description: 'Space-separated component names (keys of tools/snapshot/recipes.json), or all'
required: true
default: 'all'
release:
description: 'Release tag to publish to (created if missing)'
required: true
default: 'component-snapshots-v1'
installers_release:
description: 'Release the installers are downloaded from'
required: true
default: 'system-libraries-v1'
permissions:
contents: write
jobs:
record:
runs-on: ubuntu-24.04
timeout-minutes: 300
steps:
- uses: actions/checkout@v4
- name: Install Wine and the extractors
run: |
set -euo pipefail
sudo dpkg --add-architecture i386
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends wine wine64 wine32:i386 xvfb p7zip-full \
cmake build-essential libboost-iostreams-dev libboost-filesystem-dev libboost-date-time-dev \
libboost-system-dev libboost-program-options-dev liblzma-dev >/dev/null
wine --version
# innoextract from its main branch: the 1.9 release does not read Inno Setup 6.4 (K-Lite).
git clone -q --depth 1 https://github.com/dscharrer/innoextract.git /tmp/innoextract
cmake -S /tmp/innoextract -B /tmp/innoextract/build -DCMAKE_BUILD_TYPE=Release -DUSE_LZMA=ON >/dev/null
cmake --build /tmp/innoextract/build -j"$(nproc)" >/dev/null
sudo install -m755 /tmp/innoextract/build/innoextract /usr/local/bin/innoextract
innoextract --version | head -1
- name: Record
env:
GH_TOKEN: ${{ github.token }}
run: |
set -uo pipefail
mkdir -p installers out
names='${{ inputs.components }}'
if [ "$names" = all ]; then
names=$(python3 -c "import json; print(' '.join(k for k in json.load(open('tools/snapshot/recipes.json')) if k != '_'))")
fi
# Every asset the chosen recipes and their prerequisites name, downloaded once.
python3 - $names > assets.txt <<'PY'
import json, sys
r = json.load(open("tools/snapshot/recipes.json"))
wanted = set()
def add(name):
rec = r[name]
for item in rec.get("installers") or [{"asset": rec.get("installer")}]:
wanted.add(item["asset"])
for dep in rec.get("after", []):
add(dep)
for name in sys.argv[1:]:
add(name)
print("\n".join(sorted(wanted)))
PY
while read -r asset; do
[ -n "$asset" ] && gh release download "${{ inputs.installers_release || 'system-libraries-v1' }}" -p "$asset" -D installers --clobber
done < assets.txt
ls -l installers
failed=""
for name in $names; do
echo "::group::$name"
if ! python3 tools/snapshot/record.py --component "$name" --installers-dir installers --out out; then
echo "::warning::$name: the recording failed"
failed="$failed $name"
fi
echo "::endgroup::"
done
ls -l out
{ echo "# Recordings"; echo; for f in out/*.summary.md; do sed -n '1,14p' "$f"; echo; done; [ -n "$failed" ] && echo "**Failed:**$failed"; } >> "$GITHUB_STEP_SUMMARY"
echo "FAILED=$failed" >> "$GITHUB_ENV"
- name: Publish
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
tag='${{ inputs.release || 'component-snapshots-v1' }}'
if ! gh release view "$tag" >/dev/null 2>&1; then
gh release create "$tag" --title "Component snapshots" --notes "What each Windows component's installer leaves behind in a Wine prefix, recorded by tools/snapshot/record.py: the registry values it writes and the files it places, mapped back into the vendor's own installer. Read by the apps' Windows components page; one <name>.snapshot.json and <name>.summary.md per component." --latest=false
fi
gh release upload "$tag" out/*.snapshot.json out/*.summary.md $(ls out/*.files.tar.xz 2>/dev/null) --clobber
[ -z "${FAILED:-}" ] || { echo "::error::recordings failed:$FAILED"; exit 1; }
- name: Keep the recordings as an artifact too
uses: actions/upload-artifact@v4
with:
name: snapshots
path: out/
+42
View File
@@ -0,0 +1,42 @@
name: Mirror ARM64 AppImages (SteamDeck)
# Tier-2 emulators: upstream's own aarch64 Linux builds, copied to our release so the catalog can
# point at a URL that does not move and a checksum that was taken here. Nothing is modified.
on:
workflow_dispatch:
inputs:
release:
description: 'Release tag to publish to'
required: true
default: 'steamdeck-desktop-r1'
permissions:
contents: write
jobs:
mirror:
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- name: Fetch and publish
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
run: |
set -euo pipefail
mkdir -p out && cd out
# id upstream url. Only what is being added: every file listed is re-fetched and replaces
# the release copy, and a moving upstream link (DuckStation's "latest") would no longer
# match its checksum in desktop.json. Mirrored before: rpcs3 (build 9e86f165), duckstation
# (latest @ 2026-09-20), melonds (1.1), armsx2 (nightly-20260923, 4K pages).
while read -r id url; do
[ -n "$id" ] || continue
ext=${url##*.}
curl -fsSL --retry 5 --retry-delay 10 -o "$id.$ext" "$url"
sha256sum "$id.$ext" | awk '{print $1}' > "$id.sha256"
echo " $id $(stat -c %s "$id.$ext") $(cat "$id.sha256") $id.$ext"
done <<'LIST'
armsx2-16k https://github.com/ARMSX2/ARMSX2/releases/download/nightly-20260923/ARMSX2-nightly-20260923-92aa2174da-Linux-arm64-16K-pages.AppImage
LIST
gh release upload '${{ inputs.release }}' ./* --clobber
+185
View File
@@ -0,0 +1,185 @@
name: Mirror a release
# Brings a release's assets into this repo by CI: every asset is downloaded from the source
# release, checked against what the catalog that points at it promises (md5/size from the Bottles
# manifests in components.json, sha256/size in desktop.json, linuxfs.json and steam-seed.json,
# the files-archive sha256 inside each recording), uploaded under the same tag here, and its
# sha256 confirmed against what GitHub stored. A SHA256SUMS and a MIRROR.md naming this run go
# with it. Nothing is uploaded by hand.
on:
workflow_dispatch:
inputs:
tag:
description: 'Release tag (same on the source and here)'
required: true
source_repo:
description: 'Repository the release is mirrored from'
required: true
default: 'The412Banner/winlator-contents'
catalogs:
description: 'Space-separated catalog files (or "snapshots") that point at this release, for verification'
required: true
default: 'components.json'
title:
description: 'Release title (used when the release is created here)'
required: false
default: ''
upstream:
description: 'Fetch vendor files from their original links (Bottles manifests, pinned) rather than the source release'
type: boolean
default: true
permissions:
contents: write
concurrency:
group: system-libraries-v1
cancel-in-progress: false
env:
# The Bottles dependency manifests the catalog's checksums came from, at the commit current when
# the catalog was made (2026-06-23): their links are where the vendor files are fetched from.
BOTTLES_SHA: 8dd62af0e53f9b46e6757193a98c146384b0a0d8
jobs:
mirror:
runs-on: ubuntu-24.04
timeout-minutes: 300
steps:
- uses: actions/checkout@v4
- name: Mirror
env:
GH_TOKEN: ${{ github.token }}
SOURCE: ${{ inputs.source_repo || 'The412Banner/winlator-contents' }}
ONE_TAG: ${{ inputs.tag }}
ONE_CATALOGS: ${{ inputs.catalogs }}
ONE_TITLE: ${{ inputs.title }}
UPSTREAM: ${{ inputs.upstream }}
run: |
set -euo pipefail
# The pinned Bottles manifests, for the vendors' own links.
bottles=$(mktemp -d)
curl -fsSL "https://github.com/bottlesdevs/dependencies/archive/$BOTTLES_SHA.tar.gz" | tar -xz --strip-components=1 -C "$bottles"
echo "Bottles manifests at $BOTTLES_SHA: $(find "$bottles" -name '*.yml' | wc -l)"
# tag|catalogs|title, one per line; a dispatch mirrors just its own tag.
if [ -n "$ONE_TAG" ]; then
sets="$ONE_TAG|$ONE_CATALOGS|$ONE_TITLE"
else
sets=$(cat <<'LIST'
system-libraries-v1|components.json|Windows components (system libraries)
component-snapshots-v1|snapshots|Component snapshots
steamdeck-desktop-r1|desktop.json|Desktop and emulator packages
linuxfs-r9|linuxfs.json|Linux runtime r9
steam-proton-arm64-25502785|steam-seed.json|Steam seed: Proton Experimental (ARM64) 25502785
LIST
)
fi
failed=""
echo "$sets" | while IFS='|' read -r tag catalogs title; do
tag=$(echo "$tag" | xargs); catalogs=$(echo "$catalogs" | xargs); title=$(echo "$title" | xargs)
[ -n "$tag" ] || continue
echo "::group::$tag from $SOURCE"
work=$(mktemp -d)
assets=$(gh release view "$tag" -R "$SOURCE" --json assets -q '.assets[].name' | grep -vE '^(SHA256SUMS|MIRROR.md|BUILT.md)$' || true)
count=$(echo "$assets" | grep -c . || true)
echo "$count assets on $SOURCE/$tag"
# GitHub's stored sha256 of every asset on both sides: an asset already here with the
# same digest as the source is not moved again, so a re-run only does what is missing.
gh api "repos/$SOURCE/releases/tags/$tag" --paginate --jq '.assets[] | "\(.name) \(.digest // "")"' > "$work/source.digests"
gh api "repos/$GITHUB_REPOSITORY/releases/tags/$tag" --paginate --jq '.assets[] | "\(.name) \(.digest // "")"' > "$work/here.digests" 2>/dev/null || : > "$work/here.digests"
# Expectations: the catalogs in this checkout, or the recordings themselves.
if [ "$catalogs" = snapshots ]; then
mkdir -p "$work/snap"
echo "$assets" | grep '\.snapshot\.json$' | while read -r name; do
gh release download "$tag" -R "$SOURCE" -p "$name" -D "$work/snap" --clobber
done
python3 tools/mirror/expectations.py "$tag" "$work/snap" > "$work/expect.json"
else
python3 tools/mirror/expectations.py "$tag" $catalogs > "$work/expect.json"
fi
# Vendor links for the assets a Bottles manifest describes with the same checksum.
echo '{}' > "$work/upstream.json"
if [ "$UPSTREAM" = true ] && [ "$catalogs" != snapshots ] && echo "$catalogs" | grep -q components.json; then
python3 tools/mirror/upstream.py components.json "$bottles" "$tag" > "$work/upstream.json"
echo "$(python3 -c "import json;print(len(json.load(open('$work/upstream.json'))))") assets have a vendor link"
fi
if ! gh release view "$tag" >/dev/null 2>&1; then
gh release create "$tag" --title "${title:-$tag}" --latest=false \
--notes "Mirrored by CI from $SOURCE ($tag): $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID. Each asset was checked against the catalog that points at it (see MIRROR.md) and its sha256 confirmed after upload."
fi
: > "$work/SHA256SUMS"
{
echo "# Mirror of $SOURCE/$tag"
echo
echo "Run: $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
echo "Verified against: $catalogs"
echo
echo "| asset | size | catalog check | origin |"
echo "|---|---|---|---|"
} > "$work/MIRROR.md"
: > "$work/failed"
echo "$assets" | while read -r name; do
[ -n "$name" ] || continue
case "$name" in SHA256SUMS|MIRROR.md|BUILT.md) continue;; esac
src_digest=$(awk -v n="$name" '$1==n{print $2}' "$work/source.digests")
here_digest=$(awk -v n="$name" '$1==n{print $2}' "$work/here.digests")
vendor=$(python3 -c "import json,sys;print(json.load(open('$work/upstream.json')).get(sys.argv[1],''))" "$name")
origin="copied from $SOURCE"
if [ -n "$vendor" ]; then
# The vendor's own file; the source release is the fallback if the link has died.
if curl -fL --retry 3 --retry-delay 15 --max-time 1500 -o "$work/asset" "$vendor"; then
origin="fetched from the vendor: $(echo "$vendor" | cut -d/ -f3)"
else
echo "::warning::$name: vendor link failed ($vendor); taking the source release's copy"
origin="copied from $SOURCE (vendor link failed)"
rm -f "$work/asset"
fi
elif [ -n "$src_digest" ] && [ "$src_digest" = "$here_digest" ]; then
echo "$name: already here with the same sha256, skipped"
echo "${src_digest#sha256:} $name" >> "$work/SHA256SUMS"
echo "| $name | | already mirrored (same sha256) | |" >> "$work/MIRROR.md"
continue
fi
if [ ! -s "$work/asset" ]; then
for attempt in 1 2 3; do
gh release download "$tag" -R "$SOURCE" -p "$name" -O "$work/asset" --clobber && break
[ "$attempt" -lt 3 ] || { echo "::error::could not download $name"; exit 1; }
sleep 20
done
fi
if out=$(python3 tools/mirror/verify.py "$work/asset" "$name" "$work/expect.json"); then
check=$(echo "$out" | sed -n 2p | sed 's/^ *//')
else
echo "$out"; echo "::error::$name failed verification and was not uploaded"
echo "$name" >> "$work/failed"; rm -f "$work/asset"; continue
fi
echo "$out" | head -1
sha=$(sha256sum "$work/asset" | cut -d' ' -f1)
size=$(stat -c %s "$work/asset")
cp "$work/asset" "$work/$name"
for attempt in 1 2 3; do
gh release upload "$tag" "$work/$name" --clobber && break
[ "$attempt" -lt 3 ] || { echo "::error::could not upload $name"; exit 1; }
sleep 20
done
stored=$(gh api "repos/$GITHUB_REPOSITORY/releases/tags/$tag" --jq ".assets[] | select(.name==\"$name\") | .digest" | sed 's/^sha256://')
if [ -n "$stored" ] && [ "$stored" != "$sha" ]; then
echo "::error::$name: GitHub stored sha256 $stored, uploaded $sha"; exit 1
fi
echo "$sha $name" >> "$work/SHA256SUMS"
echo "| $name | $size | $check | $origin |" >> "$work/MIRROR.md"
rm -f "$work/$name" "$work/asset"
done
# SHA256SUMS from what GitHub holds now (covers files other workflows built onto this release).
gh api "repos/$GITHUB_REPOSITORY/releases/tags/$tag" --paginate --jq '.assets[] | select(.name!="SHA256SUMS" and .name!="MIRROR.md" and .name!="BUILT.md") | "\(.digest|sub("^sha256:";"")) \(.name)"' | sort -k2 > "$work/SHA256SUMS"
gh release upload "$tag" "$work/SHA256SUMS" "$work/MIRROR.md" --clobber
here=$(gh release view "$tag" --json assets -q '.assets[].name' | grep -vcE '^(SHA256SUMS|MIRROR.md|BUILT.md)$' || true)
nfailed=$(grep -c . "$work/failed" || true)
echo "$tag: $here of $count assets mirrored; $nfailed refused by verification"
{ echo "## $tag"; echo; echo "$here of $count assets mirrored from \`$SOURCE\`, verified against \`$catalogs\`."; [ "$nfailed" = 0 ] || { echo; echo "Refused:"; sed 's/^/- /' "$work/failed"; }; echo; } >> "$GITHUB_STEP_SUMMARY"
[ "$here" = "$count" ] && [ "$nfailed" = 0 ] || { echo "::error::$tag: $here of $count assets, $nfailed refused"; exit 1; }
rm -rf "$work"
echo "::endgroup::"
done
+11
View File
@@ -34,3 +34,14 @@ repository pins by name and sha256.
GPL-3.0, the same as DroidDeck. See [LICENSE](LICENSE). The patch series modifies Wine, which is
LGPL-2.1-or-later.
## Catalogs and mirrored releases
DroidDeck reads its package catalogs from this repository's `main`: `components.json` (Windows
components), `desktop.json` (desktop and emulator packages), `linuxfs.json` (the Linux runtime) and
`steam-seed.json` (the Steam seed). Every file they point at lives on a release here, produced by a
workflow in `.github/workflows/`: `mirror-release.yml` brings an existing release in with every
asset checked against the catalog (and leaves `SHA256SUMS` + `MIRROR.md` naming the run),
`component-snapshots.yml` records what a Windows component's installer leaves behind,
`build-desktop.yml`, `build-emulator-source.yml`, `mirror-appimages.yml` and `build-linuxfs.yml`
build the rest. Nothing on a release is uploaded by hand.
+5007
View File
File diff suppressed because it is too large. Load diff
+115
View File
@@ -0,0 +1,115 @@
{
"schemaVersion": 1,
"category": "steamdeck-desktop",
"release": "steamdeck-desktop-r1",
"note": "Hosted packages the SteamDeck app installs into its Linux runtime on request. A tar package extracts over the r9 rootfs; an appimage lands in /opt/appimages with a .desktop entry and runs extracted (proot has no FUSE). Tier 1 is packaged natively for aarch64 by Arch Linux ARM; tier 2 is upstream's own ARM64 build, mirrored here unmodified with the checksum taken at mirror time; tier 3 is built from source by us.",
"packages": [
{
"id": "desktop",
"name": "Desktop (LXQt on labwc)",
"tier": 1,
"version": "r1",
"kind": "tar",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/desktop.tar.zst",
"sha256": "c2bb860b3b1688287451e2495be5db313ec07f8c8af501dfac2768a9f417bc86",
"size": 426049032,
"notes": "panel, file manager, terminal, Firefox, mpv; Steam as a window or Big Picture from an icon"
},
{
"id": "emulators",
"name": "Emulators (PPSSPP, RetroArch + cores)",
"tier": 1,
"version": "r1",
"kind": "tar",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/emulators.tar.zst",
"sha256": "a1f4806a9663d28f424b4cab1277e0ede80a92e69de5bc063cdc8b45af220164",
"size": 727860354,
"notes": "PSP; GameCube/Wii (Dolphin), PS1, PS2 (Play!), Dreamcast, DS, N64, GBA, SNES, Genesis, MAME, ScummVM cores; DOSBox, Mednafen"
},
{
"id": "duckstation",
"name": "DuckStation (PS1)",
"tier": 2,
"version": "latest-20260920",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/duckstation.AppImage",
"sha256": "d67758c25fe3772558662e39bb508620c997cc6cd3d66ecca835e754923f158a",
"size": 85995776,
"icon": "duckstation",
"category": "Game",
"notes": "standalone, upstream ARM64 build"
},
{
"id": "melonds",
"name": "melonDS 1.1 (DS)",
"tier": 2,
"version": "1.1",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/melonds.zip",
"sha256": "c537ae018d6dcedfe9da0317a1d5c0163e69201e23abfa163fa2a63e437ace09",
"size": 33391635,
"icon": "melonDS",
"category": "Game",
"notes": "standalone, upstream ARM64 build"
},
{
"id": "rpcs3",
"name": "RPCS3 (PS3)",
"tier": 2,
"version": "0.0.42-20024",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/rpcs3.AppImage",
"sha256": "bce484ff7071badb209cfb8b061c486294e60ef776121173e129770d76efb8a5",
"size": 80330873,
"icon": "rpcs3",
"category": "Game",
"notes": "upstream ARM64 build; expect only light titles on a phone"
},
{
"id": "armsx2",
"name": "ARMSX2 (PS2, fast on ARM64)",
"tier": 2,
"version": "nightly-20260923-92aa2174da",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/armsx2.AppImage",
"sha256": "629ebcef049ebe03c0ddefd4e0667dd38e6e242fe9aa275f6cb4d19f6facf64e",
"size": 57281032,
"icon": "pcsx2",
"category": "Game",
"notes": "PCSX2 with ARM64 recompilers (github.com/ARMSX2/ARMSX2, GPL-3.0); upstream Linux arm64 4K-pages nightly, mirrored unmodified. Much faster than PCSX2 here, which interprets the PS2's CPUs on ARM64",
"pages": {
"16384": {
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/armsx2-16k.AppImage",
"sha256": "74783f471b95f72132a81e0e573235e8882e57e6653fb94cef04672f68851033",
"size": 57264648
}
}
},
{
"id": "dolphin",
"name": "Dolphin (GameCube / Wii)",
"tier": 3,
"version": "master-20260920",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/dolphin.AppImage",
"sha256": "0df02c52e89e3740858c14397b11a321a3728c818cd2755505642e1cf006da85",
"size": 85973512,
"icon": "dolphin-emu",
"category": "Game",
"notes": "built from source for ARM64; no upstream ARM64 build exists"
},
{
"id": "cemu",
"name": "Cemu (Wii U)",
"tier": 3,
"version": "main-20260924",
"kind": "appimage",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steamdeck-desktop-r1/cemu.AppImage",
"sha256": "9628000a90a91784d223fc57fad64f2662da3028b941d13221a3ff969c3590a4",
"size": 64170504,
"icon": "info.cemu.Cemu",
"category": "Game",
"notes": "built from source for ARM64 (Cemu c717fca); no upstream ARM64 build exists. Rumble is optional, so a controller without force feedback no longer stops it at startup."
}
]
}
+12
View File
@@ -0,0 +1,12 @@
{
"schemaVersion": 1,
"category": "linux-runtime",
"release": "linuxfs-r9",
"name": "Linux runtime (gamescope + native ARM Steam)",
"version": "r9",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/linuxfs-r9/linuxfs.tar.zst",
"sha256": "d3240c1531531615b0dd06b6ad56e92db9015a8641eb18c3a613835ae45992e7",
"size": 790907606,
"file_size": 790907606,
"note": "Arch Linux ARM aarch64 rootfs with gamescope 3.16, Xwayland, Mesa 26.2 (Turnip with the KGSL backend, Zink), GTK 2, PulseAudio and Bannerlator's session scripts. Installed by the app into files/linuxfs; a gamescope session refuses to start without it. r9 makes a first run reach a game unaided: it creates the ~/.steam/sdkarm64 helper path the arm64 client launches every game through (missing before, so launches died before Proton was invoked), names the ARM64 Proton up front so the client offers and fetches it, and registers that tool in-session when it lands instead of needing a restart. The session also prints a milestone per stage. From r8: the libraries Proton's GStreamer plugins need. From r7: games are told the panel's real refresh rate. From r6: installed Windows titles are set to the ARM64 Proton automatically. From r5: the same proot at opt/android-host, so the runtime installs and runs without root and an app update cannot replace it; the tool is listed as Bannerlator Proton (ARM64) so it is not mistaken for Valve's own ARM64 Protons, which need user namespaces Android does not give. Contains no Valve software - the native arm64 Steam client is fetched from Valve at first use. Built from tools/linuxfs in the Bannerlator repo; ported from WinNative's gamescope runtime (GPL-3.0)."
}
+17
View File
@@ -0,0 +1,17 @@
{
"schemaVersion": 1,
"category": "steam-seed",
"note": "Placed by DroidDeck over its Linux runtime before the Steam client's first start. Not a user-installable package; kept out of desktop.json so it never shows in the package list.",
"packages": [
{
"id": "proton-arm64",
"name": "Proton Experimental (ARM64)",
"version": "25502785",
"kind": "tar",
"url": "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/steam-proton-arm64-25502785/proton-experimental-arm64-25502785.tar.zst",
"sha256": "d6c480a90d981801be0906a3f3418eb9bd9461c30e411140951cb0daf9d2a189",
"size": 398113534,
"notes": "Valve's Proton Experimental (ARM64), depot 4427311 manifest 6971993485335684515, with appmanifest_4427310.acf (LastOwner 0). The client adopts it at first sign-in and updates it itself."
}
]
}
+274
View File
@@ -0,0 +1,274 @@
#!/usr/bin/env python3
"""Builds the DLL bundles DroidDeck's Windows components lay out, from the vendors' own packages.
A bundle is a small archive of DLLs in win32/ and win64/ folders that a catalog entry extracts
and copies into a prefix. The ones on system-libraries-v1 were made by hand in June 2026; this
rebuilds each from the vendor package named in tools/bundles/recipes.json (the Bottles manifest's
source, with its md5), by opening the package and picking the same files. The result must match
the hand-made bundle file for file (tools/bundles/expect.json, sha256 per file) before it is
packed; a difference is reported and the bundle refused, so a rebuilt bundle is never silently
different from what users have been installing.
build.py --recipes recipes.json --expect expect.json --out out/ [--only name ...] [--source-dir dir]
Sources are downloaded (or taken from --source-dir, where a mirror of the package may already
sit), md5-checked, and opened once each: .exe/.cab/.dl_ with cabextract or 7-Zip, .zip with
7-Zip, VC++ installers through DroidDeck's own unpacker (droiddeck-msi-install --unpack, which
opens burn bundles and cab-SFX wrappers the way the phone does). Archives are packed
reproducibly: sorted names, zero timestamps, no owner.
"""
import argparse
import fnmatch
import glob
import hashlib
import io
import json
import os
import shutil
import subprocess
import sys
import tarfile
import tempfile
import time
import urllib.request
import zipfile
from pathlib import Path
def sha256_of(path):
digest = hashlib.sha256()
with open(path, "rb") as source:
for block in iter(lambda: source.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
def md5_of(path):
digest = hashlib.md5()
with open(path, "rb") as source:
for block in iter(lambda: source.read(1 << 20), b""):
digest.update(block)
return digest.hexdigest()
def run(cmd, **kw):
return subprocess.run(cmd, capture_output=True, text=True, errors="replace", **kw)
class Sources:
"""Downloads and opens each vendor package once."""
def __init__(self, cache, source_dir, unpacker):
self.cache = Path(cache)
self.cache.mkdir(parents=True, exist_ok=True)
self.source_dir = Path(source_dir) if source_dir else None
self.unpacker = unpacker
self.opened = {}
self.log = []
def fetch(self, source):
name, url, md5 = source["name"], source["url"], source.get("md5", "").lower()
target = self.cache / name
if not target.is_file():
local = self.source_dir / name if self.source_dir else None
if local and local.is_file():
shutil.copyfile(local, target)
origin = "from the source mirror"
else:
for attempt in range(1, 5):
try:
request = urllib.request.Request(url, headers={"User-Agent": "DroidDeck-Components bundle builder"})
with urllib.request.urlopen(request, timeout=120) as response, open(target, "wb") as out:
shutil.copyfileobj(response, out, 1 << 20)
break
except Exception as error: # noqa: BLE001
if attempt == 4:
raise RuntimeError("could not download %s: %s" % (url, error))
time.sleep(15 * attempt)
origin = "fetched from %s" % url.split("/")[2]
actual = md5_of(target)
if md5 and actual != md5:
target.unlink()
raise RuntimeError("%s: md5 %s, the recipe says %s" % (name, actual, md5))
self.log.append("%s: %s, md5 %s%s" % (name, origin, actual, "" if md5 else " (unchecked: no recipe md5)"))
return target
def open(self, source):
"""The package's contents on disk: a folder, opened once per package."""
name = source["name"]
if name in self.opened:
return self.opened[name]
package = self.fetch(source)
folder = self.cache / (name + ".opened")
if not folder.is_dir():
folder.mkdir()
how = source.get("open", "auto")
if how == "installer":
result = run(["python3", self.unpacker, "--unpack", str(package), str(folder)])
if result.returncode != 0:
raise RuntimeError("%s: unpacker failed: %s" % (name, result.stderr[-400:]))
elif how == "install":
# DroidDeck's full layout of the installer (drive_c with real file names).
result = run(["python3", self.unpacker, str(package), str(folder)])
if result.returncode != 0:
raise RuntimeError("%s: install layout failed: %s" % (name, result.stderr[-400:]))
elif package.suffix.lower() == ".zip" or how == "zip":
with zipfile.ZipFile(package) as archive:
archive.extractall(folder)
elif how in ("auto", "cab") and run(["cabextract", "-q", "-d", str(folder), str(package)]).returncode == 0 \
and any(folder.rglob("*")):
pass
else:
result = run(["7z", "x", "-y", "-bso0", "-bsp0", "-o" + str(folder), str(package)])
if result.returncode != 0 or not any(folder.rglob("*")):
raise RuntimeError("%s: neither cabextract nor 7-Zip opened it" % name)
# Cabinets inside (DirectX's per-feature cabs, SP1's nested cab, .dl_ files): opened
# in place so a recipe can name a file by its path inside.
self.open_nested(folder, 0)
self.opened[name] = folder
return folder
def open_nested(self, folder, depth):
if depth > 2:
return
for path in sorted(p for p in folder.rglob("*") if p.is_file()):
if path.name.endswith(".opened") or path.suffix.lower() not in (".cab", ".exe", ".msi", ".dl_", ".ex_", ".zip", ".cpl"):
continue
if path.suffix.lower() in (".exe", ".cpl") and run(["cabextract", "-l", str(path)]).returncode != 0:
continue # a plain program, not a cabinet wrapper
out = path.with_name(path.name + ".opened")
if out.exists():
continue
out.mkdir()
ok = run(["cabextract", "-q", "-d", str(out), str(path)]).returncode == 0 and any(out.rglob("*"))
if not ok:
shutil.rmtree(out, ignore_errors=True)
out.mkdir()
ok = run(["7z", "x", "-y", "-bso0", "-bsp0", "-o" + str(out), str(path)]).returncode == 0 and any(out.rglob("*"))
if ok:
self.open_nested(out, depth + 1)
else:
shutil.rmtree(out, ignore_errors=True)
def find(folder, patterns, expected=None, log=None):
"""The file under folder matching one of the globs (tried in order; relative, / separated,
case-insensitive; nested '.opened' folders count as path parts). When a package holds several
copies (Windows 7 SP1 carries more than one gdiplus.dll), the one with the expected sha256 is
taken and its path logged, so the recipe can be pinned to it."""
if isinstance(patterns, str):
patterns = [patterns]
for pattern in patterns:
pattern = pattern.lower()
hits = []
for path in folder.rglob("*"):
if path.is_file():
rel = path.relative_to(folder).as_posix().lower()
if fnmatch.fnmatch(rel, pattern) or fnmatch.fnmatch(rel, "*/" + pattern):
hits.append(path)
if not hits:
continue
hits.sort(key=lambda p: (len(p.relative_to(folder).parts), str(p)))
if len(hits) > 1 and expected:
for hit in hits:
if sha256_of(hit) == expected:
if log is not None:
log.append("%s: %d copies match, took %s (the expected one)" % (pattern, len(hits), hit.relative_to(folder).as_posix()))
return hit
return hits[0]
raise RuntimeError("no file matches %s" % " | ".join(patterns))
def pack_tar(tree, out):
with tarfile.open(out, "w:xz", preset=9) as archive:
for path in sorted(p for p in Path(tree).rglob("*") if p.is_file()):
info = archive.gettarinfo(str(path), path.relative_to(tree).as_posix())
info.mtime, info.uid, info.gid, info.uname, info.gname, info.mode = 0, 0, 0, "", "", 0o644
with open(path, "rb") as source:
archive.addfile(info, source)
def pack_zip(tree, out):
with zipfile.ZipFile(out, "w", zipfile.ZIP_DEFLATED) as archive:
for path in sorted(p for p in Path(tree).rglob("*") if p.is_file()):
info = zipfile.ZipInfo(path.relative_to(tree).as_posix(), date_time=(1980, 1, 1, 0, 0, 0))
info.compress_type = zipfile.ZIP_DEFLATED
info.external_attr = 0o644 << 16
archive.writestr(info, path.read_bytes())
def build_one(name, recipe, expect, sources, out_dir):
if recipe.get("skip"):
return None, recipe["skip"]
tree = Path(tempfile.mkdtemp(prefix="bundle-"))
try:
for item in recipe["files"]:
patterns = item["from"] if isinstance(item["from"], list) else [item["from"]]
source = next(s for s in recipe["sources"] if s["name"] == patterns[0].split(":", 1)[0])
folder = sources.open(source)
src = find(folder, [p.split(":", 1)[1] for p in patterns], expect.get(name, {}).get(item["to"]), sources.log)
target = tree / item["to"]
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copyfile(src, target)
built = {p.relative_to(tree).as_posix(): sha256_of(p) for p in tree.rglob("*") if p.is_file()}
wanted = expect.get(name, {})
accepted = ""
if built != wanted:
missing = sorted(set(wanted) - set(built))
extra = sorted(set(built) - set(wanted))
differ = sorted(k for k in set(wanted) & set(built) if wanted[k] != built[k])
if recipe.get("accept_different") and not missing and not extra:
accepted = "; differs from the hand-made bundle in %d file(s), accepted: %s" % (len(differ), recipe["accept_different"])
else:
return False, "does not match the hand-made bundle: missing %s, extra %s, different %s" % (missing, extra, differ)
out = Path(out_dir) / name
if name.endswith(".tar.xz"):
pack_tar(tree, out)
elif name.endswith(".zip"):
pack_zip(tree, out)
else:
shutil.copyfile(tree / recipe["files"][0]["to"], out)
return True, "%d files, %d bytes, %s%s" % (len(built), out.stat().st_size, sha256_of(out), accepted)
finally:
shutil.rmtree(tree, ignore_errors=True)
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--recipes", required=True)
parser.add_argument("--expect", required=True)
parser.add_argument("--out", required=True)
parser.add_argument("--cache", default=None)
parser.add_argument("--source-dir", default=None)
parser.add_argument("--unpacker", default="droiddeck-msi-install")
parser.add_argument("--only", nargs="*")
args = parser.parse_args()
recipes = json.loads(Path(args.recipes).read_text("utf-8"))
expect = json.loads(Path(args.expect).read_text("utf-8"))
Path(args.out).mkdir(parents=True, exist_ok=True)
sources = Sources(args.cache or tempfile.mkdtemp(prefix="sources-"), args.source_dir, args.unpacker)
names = [n for n in recipes if n != "_" and (not args.only or n in args.only)]
results = {}
for name in names:
print("== %s" % name, flush=True)
try:
ok, note = build_one(name, recipes[name], expect, sources, args.out)
except Exception as error: # noqa: BLE001
ok, note = False, "failed: %s" % error
results[name] = (ok, note)
print(" %s %s" % ("built" if ok else ("skipped" if ok is None else "REFUSED"), note), flush=True)
print("\n== sources")
for line in sources.log:
print(" " + line)
Path(args.out, "BUILT.md").write_text(
"# DLL bundles built from the vendors' packages\n\n| bundle | result |\n|---|---|\n" +
"".join("| %s | %s %s |\n" % (n, "built:" if ok else ("skipped:" if ok is None else "REFUSED:"), note) for n, (ok, note) in results.items()) +
"\n## Sources\n\n" + "".join("- %s\n" % line for line in sources.log), "utf-8")
bad = [n for n, (ok, _) in results.items() if ok is False]
skipped = [n for n, (ok, _) in results.items() if ok is None]
print("\n%d built, %d skipped, %d refused%s" % (len(results) - len(bad) - len(skipped), len(skipped), len(bad), ": " + " ".join(bad) if bad else ""))
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())
+226
View File
@@ -0,0 +1,226 @@
{
"amstream__libs.tar.xz": {
"win32/amstream.dll": "26012b03ad7c0802a2f26bf89c6510a78f6f4ae44d5e5eed164e22db7db334f0",
"win64/amstream.dll": "fed6c36838e0a11cd0c003eeeaf8ef0e1c1f005c03a80a89165fc2da0c9693d1"
},
"atmlib__libs.tar.xz": {
"win32/atmlib.dll": "dd521f1af21c53158ceabc1b6d95a9ef978dff84410c568ed5763b601c9ed2d8"
},
"d3dcompiler_42__libs.tar.xz": {
"win32/d3dcompiler_42.dll": "6c976311406c23aa71018d274da0ecdef43b6e3a3b0b01e941a5e8e4e974386c",
"win64/d3dcompiler_42.dll": "4d61e843aa86d8801a60d01b8e872379c5b2503795a2bcbe6141978396ade00b"
},
"d3dcompiler_46__libs.tar.xz": {
"win32/d3dcompiler_46.dll": "58d9a00888af693b2a5222fe74cfded32ce83e74f85b474f1cbe5987217b5a9d",
"win64/d3dcompiler_46.dll": "b1a06f7aa52439a948a152bfd3301d9b595c78969bb77c9741bba935139f58a0"
},
"devenum__libs.tar.xz": {
"win32/devenum.dll": "ab49f2ebb9f99b640c14a4a1d830b35685aa758c7b1f5c62d77fdb6e09081387",
"win64/devenum.dll": "ab49f2ebb9f99b640c14a4a1d830b35685aa758c7b1f5c62d77fdb6e09081387"
},
"dx8vb__libs.tar.xz": {
"win32/dx8vb.dll": "b1c26cae457ea67bf2d54f8762c7ff98230935818e26f2cf9ae3350ea61dd91d"
},
"gdiplus__libs.tar.xz": {
"win32/gdiplus.dll": "c07a936009fb275d56dc451c7d05b5f5a3f4dcd985c7f02dee57d674034c602b",
"win64/gdiplus.dll": "980ea189929d95eb36e35980fff0c81f7b78de9422771fde8f4ac7a779f5bd89"
},
"gmdls__libs.tar.xz": {
"win64/drivers/gm.dls": "3229b09b9d7d9f3f4793b0d9b34fe6abc75cfa4a2503c0c90f43ff651ba7f2c0",
"win32/drivers/gm.dls": "3229b09b9d7d9f3f4793b0d9b34fe6abc75cfa4a2503c0c90f43ff651ba7f2c0"
},
"iertutil__libs.tar.xz": {
"win32/iertutil.dll": "f5cd7de4b315f318d495c4e28292fa47b96640d044502131934190ec58f3d1d5",
"win64/iertutil.dll": "89b894eccf65704d00d30ea3bd45b184bfab8345b779f9ae2be66b9fc7226f72"
},
"mediafoundation__libs.tar.xz": {
"win32/mf.dll": "045846267bcb1d9c4931a4871c887d8496e92df655936dd5d40375e09c950510",
"win64/mf.dll": "dbf0458bed308078edd82c81e6d1209e111c5bfa5d64dd068cedfd9e0a6c05ab"
},
"mfc40__libs.tar.xz": {
"win32/mfc40.dll": "b44b89aa6d5dc1790e1e6043ff2569f1ec31142e7659c42735a3e04c2256ffed",
"win32/mfc40u.dll": "02b8e9ee0b44526f74c3e6d072a2c4ab417c958774e6cbede9064ea66110e123"
},
"mfc42__libs.tar.xz": {
"win32/mfc42u.dll": "ea9227556f96415983065a476207cff2b36056b4010a358837a35c21cf32c877"
},
"msasn1__libs.tar.xz": {
"win64/msasn1.dll": "8d2b6750e90a27bfa916520cc3a52fbeac317aea40662bc0d9b71e4de34c1207"
},
"msftedit__libs.tar.xz": {
"win32/msftedit.dll": "04ed66befdb822181ebd1d84cbf0b17aaadf8455ae742f44d7adcb26ab07bdad",
"win64/msftedit.dll": "e15ed4fefc3010c213694331ddfdc03767682325c898d773ab243e2dc8b08461"
},
"msls31__libs.tar.xz": {
"win32/msls31.dll": "841fde9b24476a7ed364a3e4a1470ac9b7358bc92f29fca4a06aab557d140850"
},
"mspatcha__libs.tar.xz": {
"win32/mspatcha.dll": "202366e216cf3139ae49125a1614dc07dd2068f7a13d9570e5336f7532dd5e71"
},
"msxml6__libs.tar.xz": {
"win32/msxml6.dll": "f4174184a7420fe3d902836dc6c1ed0e54c28b5deb94ba08637df041b0926d95",
"win32/msxml6r.dll": "7ad08fdda21796679d070570713f91d798419b288412a0740e67c6b72d3febff",
"win64/msxml6.dll": "59d4bc37687ff06a26d202b284d048bdb6a14802d4e38f7923f2a1c8653e3889",
"win64/msxml6r.dll": "5647540a43d3b9f764e4d9bce0d14b5d9e17af59b394b7abfea5377e3dce699f"
},
"oalinst_dll__libs.tar.xz": {
"win64/OpenAL32.dll": "3963b06e319180700be0bcc0d027336664bb3a4975876b4a5442f10746fab5b4",
"win32/OpenAL32.dll": "606cf8b4c22c7ae00813585279ac045115907bc66d1483eb26355907021ea4b0"
},
"powershell__libs.tar.xz": {
"win32/powershell.exe": "212236f43b05c5e803b8bfb05c3137456ba5f799df9a442a3390fda755a35ea8",
"win64/powershell.exe": "f03523112548531e513cc64872d426b0752f96f653cdcd6785567c6ef65f15f7",
"profile.ps1": "9fdbd47ca304072b8444dda2f4338c61dab227248837d6952593bb2a8b95c80b"
},
"qasf__libs.tar.xz": {
"win32/qasf.dll": "baf44e2a33f86d8a5ec848804afcb7771d55d42a443980fce45bfecf71374dc8",
"win64/qasf.dll": "45f6313ef38d5488cef7b051895f19446b5c8d6c1de0847911389bcc84d0a608"
},
"qcap__libs.tar.xz": {
"win32/qcap.dll": "aff680e62d989a62cbdec2bf70b6d17f7615f9826eaee0c8a524af80f9ff862d",
"win64/qcap.dll": "44799a5f70c8610fdf371b159c7f7035cbf97038eed09e8bce0bdb90b63b0d9e"
},
"qdvd__libs.tar.xz": {
"win32/qdvd.dll": "b6a8105e4432e1778db2a832e1f93bc34f0c59ef955945dd8bbae835278598b9",
"win64/qdvd.dll": "7218f46b6b42bce78b2e1524cb21d9622bfe9b7bf544fb95aa17df373202fa94"
},
"qedit__libs.tar.xz": {
"win32/qedit.dll": "38fd8f09cee1bb17782c6d54222b32f41acbd4ea62af4104e5f6efc2c27cbc99",
"win64/qedit.dll": "e90693ecc0c5e146b13ef65c33ca1587b2d923a0f3cbbd15bcb65f17f8f9795f"
},
"quartz__libs.tar.xz": {
"win32/quartz.dll": "fa52a0d0647413deeef57c5cb632f73a97a48588c16877fc1cc66404c3c21a2b",
"win64/quartz.dll": "701e6047d6cc30feada589eb65cefa95fd708031dcb8417d78ef2639ffa3efdd"
},
"riched20__libs.tar.xz": {
"win32/riched20.dll": "c741226a0465a8c4edcbe0f3af54de02e21931122afe3a4dad8d49553fececcc"
},
"sqlite3.zip": {
"sqlite3.dll": "ec0debfaaa036fe1421f09da809848820f9a36d35edf09317f81a776379bc42b",
"sqlite3.def": "02e76d26e9afcb06313980ce577144a1d928ed39c4d189ad8652dca34712fc77"
},
"urlmon__libs.tar.xz": {
"win32/urlmon.dll": "669bed2f3a2cffb5e25fe6cf6956e61036fa1d9127ae59f05b22cfd57cea3e2a",
"win64/urlmon.dll": "2590e88cab52fcc1b24cb262d293131c6280a5f234e0c130e77aa8697efa3b5f"
},
"vcredist2005_dll__libs.tar.xz": {
"win64/msvcm80.dll": "835195907e9f3731a07e590e4bf15a3ebce17f53d3bb6cd6ef7cfb26fbde1f55",
"win64/msvcp80.dll": "40f6502679cee0b657b0005278fbe7213bdda6deaacf868058e17737c182e1b4",
"win64/msvcr80.dll": "690f12c490bee2bf17ab7b6804e6e9b96f51c304350ccde80fe5c7eefa89720e",
"win32/msvcm80.dll": "7dcb3c45938d31854e46b5e5b0e16d538e29230d1bc81086d40c8db3bdf510bc",
"win32/msvcp80.dll": "372af797353f9335915cd06d4076bab8410775dcaf2dac0593197d7c41bbffb2",
"win32/msvcr80.dll": "c3ac989c8489a23bb96400b1856f5325ffc67e844f04651ea5d61bc20a991c6d"
},
"vcredist2008_dll__libs.tar.xz": {
"win64/msvcm90.dll": "47b4e80f1173532365fceedaeeb495bdaefac4d362eb62bb92f4fea589d2c104",
"win64/msvcp90.dll": "b3f4fda27430acc6d6bc1c3cbd518b9cae5ba0f22ab8917578a7f16270f94c8b",
"win64/msvcr90.dll": "5330682ae9c08e5f2e30c5e256b91028389bbbddaa8c38950df76616fca854ff",
"win32/msvcm90.dll": "7a74da389fbd10a710c294c2e914dc6f18e05f028f07958a2fa53ac44f0e4b90",
"win32/msvcp90.dll": "06918cf99ad26cd6cf106881c0d5bdb212dc0bac4549805c9f5906e3d03d152c",
"win32/msvcr90.dll": "8e7fe1a1f3550c479ffd86a77bc9d10686d47f8727025bb891d8f4f0259354c8"
},
"vcredist2010_dll__libs.tar.xz": {
"win64/msvcp100.dll": "5bb420fbe28315f2117376052bb8488ce84a3398dda65005b8ae1f792017e9a8",
"win64/msvcr100.dll": "1d1a1ae540ba132f998d60d3622f0297b6e86ae399332c3b47462d7c0f560a36",
"win32/msvcp100.dll": "d769fafa2b3232de9fa7153212ba287f68e745257f1c00fafb511e7a02de7adf",
"win32/msvcr100.dll": "60c06e0fa4449314da3a0a87c1a9d9577df99226f943637e06f61188e5862efa"
},
"vcredist2012_dll__libs.tar.xz": {
"win64/msvcp110.dll": "27f394ae01d12f851f1dee3632dee3c5afa1d267f7a96321d35fd43105b035ad",
"win64/msvcr110.dll": "ae996edb9b050677c4f82d56092efdc75f0addc97a14e2c46753e2db3f6bd732",
"win32/msvcp110.dll": "c8d5572ca8d7624871188f0acabc3ae60d4c5a4f6782d952b9038de3bc28b39a",
"win32/msvcr110.dll": "b30160e759115e24425b9bcdf606ef6ebce4657487525ede7f1ac40b90ff7e49"
},
"vcredist2013_dll__libs.tar.xz": {
"win64/msvcp120.dll": "3b92e964404e3f94531e7d7c4c7419561d9eca6accd98dc3979c9e3596db444c",
"win64/msvcr120.dll": "753ac30c30aae62415cc225e3d057b8b6254afe280696e0a43f1a7c3132632a6",
"win32/msvcp120.dll": "0e535e147d253fc645ccb9406c888aee374ab1f3f9d99dc057a2b19cf401458b",
"win32/msvcr120.dll": "82225ec7e2da43a7a72a3d523698747512523afa488767ca6839c63a7a5706fa"
},
"vcredist2022_dll__libs.tar.xz": {
"win64/concrt140.dll": "b2faf3b85b23c840b654e57d5497a0ad31acd02fb01856cad4725a1715d5f78e",
"win64/msvcp140.dll": "def46aa6a8f72f27bafac0c43334419486a4d1dcdb6c479a8ef7034b3e1fa4cb",
"win64/msvcp140_1.dll": "2dd670f874562fbdca5b022df1943d70a57ba91fde559280e3a1daebe4db2380",
"win64/msvcp140_2.dll": "1d60da3ac2b06482912ca852fa7047436e6e474b4cfffa3bf77f4598cfbf454c",
"win64/msvcp140_atomic_wait.dll": "e7963645e0d1db08e300614d4c5fa7194bd8173e9ab7a5558859e6b232ed3241",
"win64/msvcp140_codecvt_ids.dll": "ae8d922b00cdd93e3ebecc37beb46c800f383ebdeb9f9e5b84e04a72428b6fb3",
"win64/vcamp140.dll": "617f6031929ec70a084a9dd1f01e037494a74d8c66703b7fb53e80547f90af28",
"win64/vccorlib140.dll": "6b8d8a76c3e6664293407553650e60b94df9aaafc7c92057ea83032bd228e44f",
"win64/vcomp140.dll": "31af29c03643f8396a6f26bcd601c6369d26493d7d78b714827ab2801bd284c7",
"win64/vcruntime140.dll": "184146852727a9db4eea06178716bec3cdbb1015c911f6b0f915b184ad7775b2",
"win64/vcruntime140_1.dll": "f4d66bdd6789aaed53341dc3153655e0ef2a9201c3652400443896a592e795b6",
"win32/concrt140.dll": "1a084b2275c02b4df2b0ae395a5312a3ad3143f5ace9ba08cfd4fc42cdf9bf61",
"win32/msvcp140.dll": "546ee2af2ffff02a34dbc1139bc6eb0eb5d67d83b3be782cfead374d29c8e01e",
"win32/msvcp140_1.dll": "c19ef5e0fc2de4207800c3e7642cb18a2bf65ec8b7bc1833922c2bd4bc8bfa8e",
"win32/msvcp140_2.dll": "8511395c8b36d0473aa1da9bd11a4cacc03511f77f837e997ca51047f29e145e",
"win32/msvcp140_atomic_wait.dll": "fc5d35783a30822a1d418ee8f08130c450f4465c1dd68ce09241ab77e33c1931",
"win32/msvcp140_codecvt_ids.dll": "79dcb9ef3986802cc9881ec80ee84d9f6a2ab075585da0f87561931821c0a9e7",
"win32/vcamp140.dll": "4c2c17577b9dbc7becd8caa0aff01b1d04534e0a1bb5712bcff8780dc092b39c",
"win32/vccorlib140.dll": "62ce34167c24c6f895da00acb5b65a8fb4ff392da5bdbeeece875ac3939df187",
"win32/vcomp140.dll": "b1df4f47c8746e49b0b7db55fc5d391a1442de04863ccba497530003182b7576",
"win32/vcruntime140.dll": "87fc734e0f2884985514edace58cf649a8ad67cb058dc7b7a4068f77af86810a"
},
"vcredist6__libs.tar.xz": {
"win32/asycfilt.dll": "de83c9d9203050b40c098e4143ef8f577aa90016c7a64d4f2931b57a4c43e566",
"win32/comcat.dll": "96d04cdfaf4f4d7b8722b139a15074975d4c244302f78034b7be65df1a92fd03",
"win32/mfc42.dll": "ec63a85030c60716acdcf060abfaa95a6a3528631622fa60e7d17fbea2f751f9",
"win32/mfc42u.dll": "5b3b68bb88be968a0c7c24887b437fac6d9081671b8d8376482168383ca30b8b",
"win32/msvcirt.dll": "647f7534aaaedffa93534e4cb9b24bfcf91524828ff0364d88973be58139e255",
"win32/msvcp60.dll": "2b6b93c2d66969eb00258e2b5ad6172decebada096e3b1b077a3380c80e4a072",
"win32/msvcrt.dll": "887eb5ce93edb7192ca3e9220f07f9ca0f94db02af5862ebcbdfcb852db99fd1",
"win32/oleaut32.dll": "baeb2f7c1b8be56738d34e1d1ddf8e0eebd3a633215dc1575e14656be38b939d",
"win32/olepro32.dll": "789c3c45eda1749bd939f4a96616e1e9ef1b7dcc62a2889f65088954c64d0938",
"win32/atla.dll": "e911ff1063d16348f353ee9001fbd43e04a3c8514028d964967d32dd87d83cca",
"win32/stdole2.tlb": "6ea525bface5467c1045c3708f339a4b92a3a273f70656e061c7f7322c56d667"
},
"vcredist6sp6_inner.exe": {
"vcredist6sp6_inner.exe": "ce05a2ad669fdc331cf43b4637f7c596c279685bc45eab005c15eb5ea495d5ea"
},
"winhttp__libs.tar.xz": {
"win32/winhttp.dll": "5b40a36e68c7b428001e7ece8de43fb5effcc8a8a63671de6735c35cfd17aed2"
},
"wininet__libs.tar.xz": {
"win32/wininet.dll": "4b01872788057ecf5a1ac6647512f5671213f76651ff8e808f1310a1240f35a2",
"win64/wininet.dll": "e3ebf44621efc6381baae0f0efc13c356dcb6ee31bb258137edb3cc3e18549b5"
},
"xaudio2.7__libs.tar.xz": {
"win32/XAudio2_2.dll": "9c7897b4ee1bcd190b1c0b7b77e64ee731d234764683a1e2286af70d86b62753",
"win32/XAPOFX1_1.dll": "5ab63c0f040fdf65e681eba4daa55ed83e89ea10c426dc2fc763da0fc94f3ace",
"win32/XAudio2_5.dll": "c596de2ab8394fb62538fef0b4657317f4ead50a6d798c5d066e25e334576c27",
"win32/XAPOFX1_3.dll": "c6956ac2ee59f71e86784138b5443de6970a1274ac161945b8a44dc1d535db84",
"win32/XAudio2_6.dll": "61b57170f7c6365714396072d22cb98746718c0f44c9f0d5c62fdb1b218639c7",
"win32/XAPOFX1_4.dll": "0aee44b12913a95840ee6431d90518b0d72c54a27392e21ee6995e2151554a06",
"win32/XAudio2_1.dll": "32a7c6a4edef46f025a4a5c64b892e29baaae948e86d9ed49e82014eec1441de",
"win32/XAPOFX1_0.dll": "8b396d275de2550af8ada6a1ff71f0f4870b51c8407e44044c2dde7ad6b754cc",
"win32/XAudio2_3.dll": "04dec9d7c68962e01efec0aac0ef7a3499bc4c16e8a41bd61fe6641da48d7dcc",
"win32/XAPOFX1_2.dll": "d1a55061bcb42f69b7cf35e2985d48e30c7a90f0bc668e90390f465b36bd0161",
"win32/XAudio2_7.dll": "d1690b602cb317f7f1e1e13e3fc5819ad8b5b38a92d812078afb1b408ccc4b69",
"win32/XAPOFX1_5.dll": "becfdcd6b16523573cb52df87aa7d993f1b345ba903d0618c3b36535c3800197",
"win32/XAudio2_0.dll": "e435b73193bdf651f7ae564eba05266595ac672db45e0e22dce92d0bcb3c6513",
"win32/XAudio2_4.dll": "bcde4317debd0052b1436a6fda60e1dcb1e308979498117fa0cb50061f38101c",
"win64/XAudio2_2.dll": "9ac81279b1625cbd9a8b59716aac63ef68d722dae586291457d24366a5f4e78b",
"win64/XAPOFX1_1.dll": "cd03a9ef13123c1b9ce3d92b9e9fd9b9682c9f66c167dd8a326e4bd0acff73f2",
"win64/XAudio2_5.dll": "36c77312026a3bd5fb928f4fe599a077b08264a82f2863875c2d4619ee133b71",
"win64/XAPOFX1_3.dll": "5f03017d3b51c1dc413952d21bf35ac660693c6f7539e8f8930ddd41d197a495",
"win64/XAudio2_6.dll": "3240447ef7bab48ec32a19f36e990bbddfdcb0f664fd9f58449f46788b5c32dd",
"win64/XAPOFX1_4.dll": "d42e7e91eafce25550f08a63240e23fbe3ddd278b852e36e4eab05bc42f5ec7d",
"win64/XAudio2_1.dll": "7c7c67e2255217d12c5c2afe53556c0f131049be7db4043aed9c1379b9ca9aa1",
"win64/XAPOFX1_0.dll": "671c24f825629381870567d726b7dbc3890d665648043c041eb1100163c49e12",
"win64/XAudio2_4.dll": "df7ddbea32acecc7fd6743002d211506ab75cb1685b4d2b708ea160c610a6f0d",
"win64/XAudio2_7.dll": "f8636ccc37bbfc84107992b60e4226eb7237417112267ed64b08f72983ac4314",
"win64/XAPOFX1_5.dll": "e47dcf74d50403b376c562e4121e359e5886e42fcc60b3fa8ba53e6826854c49",
"win64/XAudio2_0.dll": "eed9479428df33f7a9be5eca9806c2e3412509274638d13032c6c0ed1a3a9ef1",
"win64/XAudio2_3.dll": "d8e9c006edcf959f414ae7e3ff2e11af0885bc84cedc8249fdc1c5e71cdca8ba",
"win64/XAPOFX1_2.dll": "f16e5795fac7a97fc1ffb5d049c011ee5f0dcdc4c6393664b6a9d94f949fdf0f"
},
"xinput__libs.tar.xz": {
"win32/xinput1_1.dll": "69cd725c53e0302e75db20e9a3e4b33f58dceaa2e6ea4938b2733df8bc289a71",
"win32/xinput1_3.dll": "8d540d484ea41e374fd0107d55d253f87ded4ce780d515d8fd59bbe8c98970a7",
"win32/xinput1_2.dll": "9714d146a785d458f0de8fef387d82c9f8e101c02407a0cbeb06f02a69518eec",
"win32/xinput9_1_0.dll": "9c14c5afd282be3e16117ff35d9896a96bdef497c45da950a730d279bd0e0cf6",
"win64/xinput1_1.dll": "87c7f921bbf171fa04f7c2adaf8b25d4bf152d12c6b99771da42f3af07f78987",
"win64/xinput1_3.dll": "756cad002e1553cfa1a91ebe8c1b9380ffabe0b4b1916c4a4db802396ddfbef8",
"win64/xinput1_2.dll": "77055b6a144c4017acd0a8b34520cf9a84be6b9c75db40f7a708b9f4e38aa138",
"win64/xinput9_1_0.dll": "81c6dfcd7bcb4f2b2580a98c8e66a3eb624b44f48b80f42a1f8134e448171cc9"
}
}
File diff suppressed because it is too large. Load diff
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""What a release's assets must hash to, read from the catalog that points at them.
Prints one JSON object {asset name: {"sha256": ..., "md5": ..., "size": ...}} (each key optional)
for the release tag given, so the mirror workflow can refuse an asset that does not match what
the catalog promises. Catalogs: components.json (Bottles manifests: md5 as file_checksum and
file_size per step), desktop.json / steam-seed.json (packages with sha256 and size), linuxfs.json
(one package with sha256 and size). For the recordings release, each <name>.snapshot.json names
its files archive with a sha256 and size; the snapshot files themselves have no outer checksum.
"""
import json
import sys
from pathlib import Path
def basename(url):
return url.split("?")[0].rstrip("/").rsplit("/", 1)[-1]
def under(url, tag):
return "/releases/download/%s/" % tag in url
def add(out, name, **fields):
"""One candidate per catalog step: the Bottles manifests describe the vendor's original file
(its md5 and size), which is what a mirrored .exe/.msi is but not what a repacked .tar.xz is,
and two steps may describe the same asset differently. An asset passes when it matches any
of its candidates; an asset with no usable candidate is mirrored on its own sha256 alone."""
candidate = {}
for key, value in fields.items():
if value in (None, "", 0, "0"):
continue
if key == "size":
value = int(value)
elif key == "md5" and not (isinstance(value, str) and len(value) == 32):
continue
elif key == "sha256" and not (isinstance(value, str) and len(value) == 64):
continue
candidate[key] = value
if not candidate or not ({"md5", "sha256"} & set(candidate)):
out.setdefault(name, [])
return
candidates = out.setdefault(name, [])
if candidate not in candidates:
candidates.append(candidate)
def from_components(doc, tag, out):
for comp in doc.get("components", []):
for step in comp.get("steps", []):
url = step.get("url") or (step.get("environment") or {}).get("url") or ""
if not isinstance(url, str) or not under(url, tag):
continue
add(out, basename(url), md5=step.get("file_checksum") or (step.get("environment") or {}).get("file_checksum"),
size=step.get("file_size") or (step.get("environment") or {}).get("file_size"))
def from_packages(doc, tag, out):
packages = doc.get("packages") or ([doc] if doc.get("url") else [])
for pkg in packages:
url = pkg.get("url", "")
if under(url, tag):
add(out, basename(url), sha256=pkg.get("sha256"), size=pkg.get("size") or pkg.get("file_size"))
def from_snapshots(folder, out):
for path in Path(folder).glob("*.snapshot.json"):
doc = json.loads(path.read_text("utf-8"))
archive = doc.get("files_archive")
if archive and archive.get("name"):
add(out, archive["name"], sha256=archive.get("sha256"), size=archive.get("size"))
def main():
tag = sys.argv[1]
out = {}
for source in sys.argv[2:]:
path = Path(source)
if path.is_dir():
from_snapshots(path, out)
continue
doc = json.loads(path.read_text("utf-8"))
if "components" in doc:
from_components(doc, tag, out)
else:
from_packages(doc, tag, out)
json.dump(out, sys.stdout, indent=1, sort_keys=True)
if __name__ == "__main__":
main()
+11
View File
@@ -0,0 +1,11 @@
{
"_": "Vendor links for assets whose Bottles manifest (pinned 2026-06-23) points at the wrong file; the catalog carries the right checksum. The mirror fetches these from the vendor and verifies them like any other.",
"vcredist2010__vcredist_x86.exe": {
"url": "https://download.microsoft.com/download/C/6/D/C6D0FD4E-9E53-4897-9B91-836EBA2AACD3/vcredist_x86.exe",
"why": "the manifest's 2010 x86 is the original release (10.0.30319) while its x64 is SP1 (10.0.40219); this is SP1 x86"
},
"vcredist2012__vcredist_x86.exe": {
"url": "https://download.microsoft.com/download/1/6/B/16B06F60-3B20-4FF2-B699-5E9B7962F9AE/VSU_4/vcredist_x86.exe",
"why": "the manifest's 2012 x86 link is the 2010 installer; this is 2012 Update 4 x86 (11.0.61030)"
}
}
+63
View File
@@ -0,0 +1,63 @@
#!/usr/bin/env python3
"""upstream.py <components.json> <bottles manifests dir> <tag>: the vendor's own download link
for every asset of the release that a Bottles dependency manifest describes with the same md5.
Prints {asset name: url}. The manifests are checked out at a pinned commit by the workflow, so
the link is the one the catalog's checksum was taken from, not whatever the manifest says today."""
import glob
import json
import os
import re
import sys
def manifest_steps(root):
"""md5 -> (component, url) over every step of every manifest (plain line scan: the files are
simple YAML lists of steps)."""
found = {}
for path in glob.glob(os.path.join(root, "**", "*.yml"), recursive=True):
name = os.path.basename(path)[:-4]
current = {}
def flush():
if current.get("url") and current.get("md5"):
found.setdefault(current["md5"].lower(), (name, current["url"]))
for line in open(path, encoding="utf-8", errors="replace"):
if re.match(r"\s*-\s*action:", line):
flush()
current = {}
match = re.match(r"\s*url:\s*(\S+)", line)
if match:
current["url"] = match.group(1)
match = re.match(r"\s*file_checksum:\s*(\S+)", line)
if match:
current["md5"] = match.group(1)
flush()
return found
def main():
catalog, root, tag = sys.argv[1:4]
doc = json.load(open(catalog, encoding="utf-8"))
steps = manifest_steps(root)
out = {}
for comp in doc.get("components", []):
for step in comp.get("steps", []):
env = step.get("environment") or {}
url = step.get("url") or env.get("url") or ""
if "/releases/download/%s/" % tag not in str(url):
continue
md5 = str(step.get("file_checksum") or env.get("file_checksum") or "").lower()
asset = url.rsplit("/", 1)[-1]
if len(md5) == 32 and md5 in steps and asset not in out:
out[asset] = steps[md5][1]
overrides = os.path.join(os.path.dirname(os.path.abspath(__file__)), "upstream-overrides.json")
if os.path.isfile(overrides):
for asset, entry in json.load(open(overrides, encoding="utf-8")).items():
if asset != "_" and isinstance(entry, dict) and entry.get("url"):
out[asset] = entry["url"]
json.dump(out, sys.stdout, indent=1, sort_keys=True)
if __name__ == "__main__":
main()
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env python3
"""verify.py <file> <asset name> <expectations.json>: exit 0 when the file matches one of the
catalog's candidates for that asset (md5 or sha256, and size when given), or when the catalog
has none; exit 1 on a mismatch. Prints the file's sha256, md5 and size either way."""
import hashlib
import json
import sys
from pathlib import Path
def main():
path, name, expectations = Path(sys.argv[1]), sys.argv[2], json.loads(Path(sys.argv[3]).read_text("utf-8"))
sha, md5 = hashlib.sha256(), hashlib.md5()
with path.open("rb") as source:
for block in iter(lambda: source.read(1 << 20), b""):
sha.update(block)
md5.update(block)
size = path.stat().st_size
actual = {"sha256": sha.hexdigest(), "md5": md5.hexdigest(), "size": size}
candidates = expectations.get(name, [])
print("%s sha256=%s md5=%s size=%d" % (name, actual["sha256"], actual["md5"], size))
if not candidates:
print(" no catalog checksum for %s: mirrored on its own sha256" % name)
return 0
for candidate in candidates:
if all(actual.get(key) == str(value).lower() for key, value in candidate.items() if key in ("md5", "sha256")):
# The hash is the check; a catalog size that disagrees with a matching hash is the
# catalog's typo (the Bottles manifests carry a few), noted but not fatal.
if "size" in candidate and candidate["size"] != size:
print(" matches the catalog's %s; the catalog's size %d is wrong (file is %d)" % (
"sha256" if "sha256" in candidate else "md5", candidate["size"], size))
else:
print(" matches the catalog (%s)" % ", ".join("%s=%s" % (k, str(v)[:12]) for k, v in candidate.items()))
return 0
print("::error::%s does not match any catalog candidate: %s" % (name, candidates), file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+2
View File
@@ -0,0 +1,2 @@
__pycache__/
*.pyc
+260
View File
@@ -0,0 +1,260 @@
{
"_": "How each installer is run unattended in a fresh Wine prefix to record what it leaves behind (tools/snapshot/record.py). 'installer'+'args', or 'installers' [{asset, args}] run in order, name assets on the system-libraries-v1 release. 'arch' = prefix: win64, or win32 for installers that refuse a 64-bit prefix (.NET 2.0/3.5, Jet); a win32 recording is laid out as on 64-bit Windows unless 'remap' is false. 'timeout' seconds per installer. 'winver' = Windows version set before recording (not recorded). 'after' = recipes installed first, so this one records only its own delta (the app installs them first too). 'host_files' = the placed files travel with the recording (needed when no extractor opens the installer; only where the licence allows, 'licence' says why). 'dll_overrides' = Wine overrides for the run itself (never recorded). 'extract_to' + 'register' = instead of running a wizard with no silent mode, unpack the installer into that drive_c folder and regsvr32 those files (paths under drive_c). 'pre_reg_delete' = registry keys removed before recording (a fresh Wine prefix may claim a .NET it does not have).",
"K-Lite": {
"installer": "K-Lite.exe",
"args": [
"/VERYSILENT",
"/SUPPRESSMSGBOXES",
"/NORESTART",
"/SP-"
],
"arch": "win64",
"timeout": 900,
"host_files": true,
"licence": "The pack's installer is Inno Setup 6.4.2, which no extractor reads yet, so the placed files travel with the recording. What this edition places is open source (LAV Filters, MPC-HC, MPC Video Renderer, DirectVobSub: GPL) plus Icaros, which its author distributes freely.",
"notes": "The pack registers its DirectShow filters with regsvr32 during setup; those CLSID / filter entries are the point of the recording."
},
"lavfilters741": {
"installer": "lavfilters741.exe",
"args": [
"/VERYSILENT",
"/SUPPRESSMSGBOXES",
"/NORESTART",
"/SP-"
],
"arch": "win64",
"timeout": 600,
"host_files": true,
"licence": "LAV Filters is GPL-2.0; the files travel with the recording so a device needs no Inno Setup extractor."
},
"lavfilters702": {
"installer": "lavfilters702.exe",
"args": [
"/VERYSILENT",
"/SUPPRESSMSGBOXES",
"/NORESTART",
"/SP-"
],
"arch": "win64",
"timeout": 600,
"host_files": true,
"licence": "LAV Filters is GPL-2.0; the files travel with the recording so a device needs no Inno Setup extractor."
},
"ffdshow": {
"installer": "ffdshow.exe",
"args": [
"/VERYSILENT",
"/SUPPRESSMSGBOXES",
"/NORESTART",
"/SP-"
],
"arch": "win64",
"timeout": 600,
"host_files": true,
"licence": "ffdshow is GPL-2.0; the files travel with the recording so a device needs no Inno Setup extractor."
},
"dirac": {
"installer": "dirac.exe",
"arch": "win32",
"timeout": 600,
"extract_to": "Program Files/Dirac",
"register": [
"Program Files/Dirac/DiracSplitter-Dirac.ax",
"Program Files/Dirac/DiracSplitter-libschroedinger.ax"
],
"notes": "NSIS wizard with no working silent mode (winetricks clicks through it with AutoHotkey): unpacked into Program Files/Dirac and the two filters registered with regsvr32, in a win32 prefix laid out as on 64-bit Windows. 7-Zip opens the installer on the device. The filters link the VC++ 2008 runtime, installed first (and a catalog dependency).",
"after": [
"vcredist2008"
]
},
"quicktime72": {
"installer": "quicktime72.exe",
"args": [
"/qn",
"ALLUSERS=1",
"DESKTOP_SHORTCUTS=0",
"QTTaskRunFlags=0",
"QTINFO.BISQTPRO=1",
"SCHEDULE_ASUW=0",
"REBOOT_REQUIRED=No"
],
"arch": "win64",
"timeout": 900,
"dll_overrides": "winemenubuilder.exe=d",
"notes": "Apple's redistribution terms do not allow hosting the files: a device takes them from the installer (a cabinet wrapper 7-Zip and cabextract open)."
},
"oalinst": {
"installer": "oalinst.exe",
"args": [
"/s"
],
"arch": "win64",
"timeout": 300,
"notes": "NSIS; the DLL-copy twin oalinst_dll already covers most games."
},
"VulkanRT": {
"installer": "VulkanRT.exe",
"args": [
"/S"
],
"arch": "win64",
"timeout": 300,
"notes": "NSIS (the catalog's /passive is an MSI switch and shows the wizard)."
},
"vcredist6sp6": {
"installer": "vcredist6sp6_inner.exe",
"args": [
"/Q"
],
"arch": "win64",
"timeout": 300,
"host_files": true,
"licence": "Microsoft Visual C++ 6 runtime redistributable: licensed for redistribution with applications."
},
"jet40": {
"installer": "jet40.exe",
"args": [
"/q"
],
"arch": "win32",
"timeout": 600,
"host_files": true,
"licence": "Microsoft Jet 4.0 SP8 redistributable: licensed for redistribution with applications.",
"notes": "32-bit only IExpress package (jetsetup.exe inside, INF-driven); /q as winetricks runs it; recorded in a win32 prefix and laid out as on 64-bit Windows."
},
"mdac28": {
"installer": "mdac28.EXE",
"args": [
"/q",
"/C:setup /qnt"
],
"arch": "win64",
"winver": "win2k",
"timeout": 900,
"host_files": true,
"licence": "Microsoft Data Access Components 2.8 redistributable: licensed for redistribution with applications.",
"notes": "The setup refuses to run on anything after Windows 2000, so the recording's prefix says win2k (the catalog's win98 is the 32-bit Wine convention)."
},
"dotnet20": {
"installer": "dotnet20__dotnetfx.exe",
"args": [
"/q",
"/c:install.exe /q"
],
"arch": "win32",
"winver": "winxp",
"timeout": 1800,
"host_files": true,
"licence": "Microsoft .NET Framework 2.0 redistributable: licensed for redistribution with applications.",
"notes": "32-bit only installer (the x64 one refuses Wine); recorded in a win32 prefix and laid out as on 64-bit Windows."
},
"dotnet20sp1": {
"installer": "dotnet20sp1__NetFx20SP1_x86.exe",
"args": [
"/q"
],
"arch": "win32",
"winver": "win2k",
"after": [
"dotnet20"
],
"timeout": 600,
"host_files": true,
"licence": "Microsoft .NET Framework 2.0 SP1 redistributable: licensed for redistribution with applications.",
"notes": "32-bit only; recorded on top of dotnet20 in a win32 prefix and laid out as on 64-bit Windows. As winetricks does: Windows 2000 for the installer, ngen/regsvcs/mscorsvw stubbed.",
"dll_overrides": "ngen.exe,regsvcs.exe,mscorsvw.exe=b",
"pre_reg_delete": [
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.0",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.5",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v4"
]
},
"dotnet35": {
"installer": "dotnet35.exe",
"args": [
"/lang:ENU",
"/q",
"/norestart"
],
"arch": "win32",
"winver": "winxp",
"timeout": 3000,
"host_files": true,
"licence": "Microsoft .NET Framework 3.5 redistributable: licensed for redistribution with applications.",
"notes": "The full redistributable (carries 2.0 SP2 and 3.0 SP2); 32-bit only, recorded in a win32 prefix and laid out as on 64-bit Windows. As winetricks does: mscoree/mscorwks native (no Wine stand-ins), Windows XP.",
"dll_overrides": "mscoree,mscorwks=n",
"pre_reg_delete": [
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.0",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.5",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v4"
]
},
"dotnet35sp1": {
"installer": "dotnet35sp1.exe",
"args": [
"/lang:ENU",
"/q",
"/norestart"
],
"arch": "win32",
"winver": "winxp",
"timeout": 3000,
"host_files": true,
"licence": "Microsoft .NET Framework 3.5 SP1 redistributable: licensed for redistribution with applications.",
"notes": "The full redistributable (carries 2.0 SP2 and 3.0 SP2); 32-bit only, recorded in a win32 prefix and laid out as on 64-bit Windows. As winetricks does: mscoree/mscorwks native (no Wine stand-ins), Windows XP.",
"dll_overrides": "mscoree,mscorwks=n;ngen.exe=n",
"pre_reg_delete": [
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.0",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v3.5",
"HKLM\\Software\\Microsoft\\NET Framework Setup\\NDP\\v4"
]
},
"aairruntime": {
"installer": "aairruntime.exe",
"args": [
"-silent",
"-eulaAccepted"
],
"arch": "win64",
"timeout": 600,
"notes": "Adobe's terms do not allow hosting the files; the installer is an archive 7-Zip can open."
},
"art2kmin": {
"installer": "art2kmin.exe",
"args": [
"/q"
],
"arch": "win32",
"timeout": 900,
"notes": "Access 2000 runtime: a 7-Zip-openable wrapper around Setup.exe + Data1.msi. Not recorded: the app's MSI engine lays Data1.msi out directly once art2kmin leaves its setup-program list."
},
"webview2": {
"installer": "webview2.exe",
"args": [
"/silent",
"/install"
],
"arch": "win64",
"timeout": 1200,
"notes": "Microsoft Edge WebView2 Evergreen standalone installer. Recorded once: 656 files, 576 of them inside a payload no extractor opens, and Microsoft's terms do not allow hosting them. Not linkable; left for a runtime that can run the installer."
},
"gfw": {
"installer": "gfw.exe",
"args": [
"/nodotnet",
"/q"
],
"arch": "win64",
"timeout": 1200,
"notes": "Games for Windows Live client + redistributable (gfwlivesetupmin, two MSIs inside). The files come from the installer on the device; signing in needs Microsoft's live servers, which nothing here records."
},
"vcredist2008": {
"installer": "vcredist2008__vcredist_x86.exe",
"args": [
"/q"
],
"arch": "win32",
"timeout": 600,
"notes": "Prerequisite only (dirac's filters link msvcr90): the app installs vcredist2008 through its own MSI engine; this recipe exists so a recording can be made on top of it."
}
}
+707
View File
@@ -0,0 +1,707 @@
#!/usr/bin/env python3
"""Records what a Windows installer leaves behind in a Wine prefix, without keeping the installer's files.
record.py --component NAME --installer FILE --out DIR [--recipes recipes.json] [--extract DIR]
A fresh 64-bit prefix is made, its files and registry noted, the installer run unattended (the
switches in recipes.json), and the prefix noted again. The difference is the recording:
<out>/<component>.snapshot.json
"files": every file the installer placed under drive_c, with size and sha256, and where
the same bytes sit inside the installer when an extractor (innoextract, 7-Zip)
can reach them ("source"), so a device pulls them from the vendor's own
installer instead of a copy we host. A file the installer generated (no
matching bytes inside it) is carried inline when small ("data", base64), else
listed as "missing" for the recipe to deal with.
"registry": every value the installer added or changed, in the shape droiddeck-wincomponents
writes into a prefix (hive, key, name, type, data), the user's folder rewritten
to Proton's steamuser and Wine's own bookkeeping left out.
<out>/<component>.summary.md a readable account of the run.
The point of the recording is the registry: a self-registering DLL (a DirectShow filter) decides
its own CLSID and filter entries at run time, so nothing but running it tells us what they are.
"""
import argparse
import base64
import hashlib
import json
import os
import re
import shutil
import subprocess
import sys
import tempfile
import time
from pathlib import Path
INLINE_FILE_LIMIT = 256 * 1024
INLINE_TOTAL_LIMIT = 2 * 1024 * 1024
# Folders whose contents are the installer's own litter, not the product.
SKIP_DIRS = ("windows/temp", "windows/installer", "users/*/temp", "users/*/appdata/local/temp",
"users/*/appdata/local/microsoft/windows/inetcache", "programdata/microsoft/windows/start menu",
"users/*/appdata/roaming/microsoft/windows/start menu", "users/*/desktop", "users/*/start menu",
"windows/logs", "windows/prefetch")
SKIP_FILE_SUFFIXES = (".log", ".lnk", ".url", ".tmp")
# Registry keys Wine or the prefix maintain by themselves between two snapshots.
SKIP_KEY_PREFIXES = (
"software\\wine\\", "software\\microsoft\\windows\\currentversion\\explorer\\",
"software\\microsoft\\windows nt\\currentversion\\profilelist", "system\\currentcontrolset\\control\\session manager\\environment\\",
"software\\microsoft\\windows\\currentversion\\installer\\userdata", "software\\microsoft\\cryptography\\",
"software\\microsoft\\windows\\currentversion\\shell extensions\\cached", "software\\classes\\local settings\\",
"software\\microsoft\\windows\\currentversion\\explorer", "software\\microsoft\\rpc\\",
"software\\microsoft\\windows nt\\currentversion\\fonts", "software\\microsoft\\windows nt\\currentversion\\winlogon",
"control panel\\", "environment", "volatile environment", "software\\microsoft\\windows\\currentversion\\uninstall\\{",
"console", "console\\",
)
SKIP_VALUE_NAMES = {"installdate", "installtime", "lastwritetime", "installsource", "sourcelist", "lastusedsource",
"estimatedsize", "modified", "installlocation"}
def sha256_of(path):
digest = hashlib.sha256()
with open(path, "rb") as source:
for chunk in iter(lambda: source.read(1 << 20), b""):
digest.update(chunk)
return digest.hexdigest()
def run(cmd, env=None, timeout=None, check=False):
return subprocess.run(cmd, env=env, timeout=timeout, check=check, capture_output=True, text=True, errors="replace")
# ---------------------------------------------------------------- files
SKIP_DIR_PATTERNS = [re.compile("^" + re.escape(p).replace("\\*", "[^/]+") + "(/|$)") for p in SKIP_DIRS]
def skipped_path(rel):
lower = rel.lower()
return lower.endswith(SKIP_FILE_SUFFIXES) or any(pattern.match(lower) for pattern in SKIP_DIR_PATTERNS)
def snapshot_files(drive_c):
found = {}
for root, dirs, files in os.walk(drive_c):
for name in files:
path = Path(root) / name
if path.is_symlink():
continue
rel = path.relative_to(drive_c).as_posix()
if skipped_path(rel):
continue
try:
found[rel] = (path.stat().st_size, sha256_of(path))
except OSError:
continue
return found
# ---------------------------------------------------------------- registry
def unescape(text):
out, i = [], 0
while i < len(text):
ch = text[i]
if ch == "\\" and i + 1 < len(text):
nxt = text[i + 1]
if nxt == "x" and i + 5 < len(text):
try:
out.append(chr(int(text[i + 2:i + 6], 16)))
i += 6
continue
except ValueError:
pass
out.append({"n": "\n", "r": "\r", "0": "\0", "t": "\t"}.get(nxt, nxt))
i += 2
else:
out.append(ch)
i += 1
return "".join(out)
def parse_reg(path):
"""A Wine registry file as {key: {name: raw}}; name "" is the default value; a key with no
values is {}. Multi-line hex values are joined."""
keys = {}
current = None
pending = None
try:
lines = Path(path).read_text("utf-8", errors="replace").split("\n")
except FileNotFoundError:
return keys
for raw_line in lines:
line = raw_line.rstrip("\r")
if pending is not None:
pending += line.strip()
if pending.endswith("\\"):
pending = pending[:-1]
continue
name, raw = pending_name, pending
keys[current][name] = raw
pending = None
continue
if not line or line.startswith(";") or line.startswith("#"):
continue
if line.startswith("["):
end = line.rfind("]")
key = line[1:end].replace("\\\\", "\\")
current = key
keys.setdefault(current, {})
continue
if current is None:
continue
if line.startswith("@="):
name, raw = "", line[2:]
elif line.startswith('"'):
end = 1
while end < len(line):
if line[end] == "\\":
end += 2
continue
if line[end] == '"':
break
end += 1
name = unescape(line[1:end])
raw = line[end + 2:] if line[end + 1:end + 2] == "=" else ""
else:
continue
if raw.endswith("\\"):
pending, pending_name = raw[:-1], name
continue
keys[current][name] = raw
return keys
def decode(raw):
"""A Wine registry value as (type, data) in droiddeck-wincomponents' vocabulary, or None."""
if raw.startswith('"') and raw.endswith('"'):
return "sz", unescape(raw[1:-1])
if raw.startswith("dword:"):
try:
return "dword", int(raw[6:], 16)
except ValueError:
return None
if raw.startswith("str(2):") or raw.startswith("str(7):"):
text = unescape(raw[7:].strip('"'))
return ("expand_sz", text) if raw.startswith("str(2)") else ("multi_sz", [s for s in text.split("\0") if s])
match = re.match(r"hex(\((\d+)\))?:(.*)$", raw)
if match:
kind = int(match.group(2) or 3)
data = bytes.fromhex(match.group(3).replace(",", "").replace("\\", ""))
if kind in (2, 7, 1):
text = data.decode("utf-16-le", errors="replace")
if kind == 7:
return "multi_sz", [s for s in text.split("\0") if s]
return ("expand_sz" if kind == 2 else "sz"), text.rstrip("\0")
if kind == 4 and len(data) == 4:
return "dword", int.from_bytes(data, "little")
return "binary", data.hex()
return None
def skipped_key(key):
lower = key.lower()
return any(lower.startswith(prefix) or lower == prefix.rstrip("\\") for prefix in SKIP_KEY_PREFIXES) or "\\volatile" in lower
def diff_registry(before, after, hive, home_user):
"""The values in [after] that [before] lacks or has differently, as registry.json entries."""
entries = []
home_pattern = re.compile(re.escape("C:\\users\\" + home_user), re.IGNORECASE) if home_user else None
for key, values in after.items():
if skipped_key(key):
continue
old = before.get(key)
if old is None and not values:
entries.append({"hive": hive, "key": key, "type": "key"})
continue
for name, raw in values.items():
if name.lower() in SKIP_VALUE_NAMES:
continue
if old is not None and old.get(name) == raw:
continue
decoded = decode(raw)
if decoded is None:
continue
kind, data = decoded
if isinstance(data, str):
if "Z:\\" in data or "z:\\" in data:
continue
if home_pattern:
data = home_pattern.sub("C:\\\\users\\\\steamuser", data)
elif isinstance(data, list) and home_pattern:
data = [home_pattern.sub("C:\\\\users\\\\steamuser", item) for item in data]
entries.append({"hive": hive, "key": key, "name": name, "type": kind, "data": data})
return entries
# ---------------------------------------------------------------- 32-bit recordings in the 64-bit layout
# A 32-bit installer that refuses a 64-bit prefix (.NET 2.0 and 3.5, Jet) is recorded in a win32
# prefix, then translated into what the same installer leaves on 64-bit Windows: files move the
# way the file-system redirector moves them (system32 -> syswow64, Program Files -> Program
# Files (x86)), and the registry is replayed into a fresh 64-bit prefix through Wine's own 32-bit
# regedit, so Wine's WOW64 registry redirection (Wow6432Node, the Classes subkeys) decides where
# each key lands rather than a hand-written table.
SYSTEM32_SHARED = ("catroot", "catroot2", "driverstore", "drivers", "etc", "logfiles", "spool")
def remap_file(rel):
parts = rel.split("/")
lower = [p.lower() for p in parts]
if lower[:2] == ["windows", "system32"] and not (len(lower) > 2 and lower[2] in SYSTEM32_SHARED):
return "/".join(["windows", "syswow64"] + parts[2:])
if lower[:1] == ["program files"]:
return "/".join(["Program Files (x86)"] + parts[1:])
if lower[:2] == ["programdata", "microsoft"] or lower[:1] == ["users"]:
return rel
return rel
def reg_escape(text):
return text.replace("\\", "\\\\").replace('"', '\\"')
def encode_value(kind, data):
if kind == "sz":
return '"%s"' % reg_escape(data)
if kind == "dword":
return "dword:%08x" % int(data)
if kind == "expand_sz":
return "hex(2):" + ",".join("%02x" % b for b in (data + "\0").encode("utf-16-le"))
if kind == "multi_sz":
return "hex(7):" + ",".join("%02x" % b for b in ("\0".join(list(data) + [""]) + "\0").encode("utf-16-le"))
if kind == "binary":
return "hex:" + ",".join(data[i:i + 2] for i in range(0, len(data), 2))
return None
def write_reg(entries, path):
"""The recorded values as a .reg file regedit imports (UTF-16, as Windows writes them)."""
roots = {"HKLM": "HKEY_LOCAL_MACHINE", "HKCU": "HKEY_CURRENT_USER"}
lines = ["Windows Registry Editor Version 5.00", ""]
by_key = {}
for entry in entries:
by_key.setdefault((entry["hive"], entry["key"]), []).append(entry)
for (hive, key), values in by_key.items():
lines.append("[%s\\%s]" % (roots[hive], key))
for entry in values:
if entry.get("type") == "key":
continue
raw = encode_value(entry["type"], entry["data"])
if raw is None:
continue
lines.append(("@=%s" if entry["name"] == "" else '"%s"=%%s' % reg_escape(entry["name"])) % raw if entry["name"] else "@=" + raw)
lines.append("")
Path(path).write_text("\n".join(lines) + "\n", "utf-16")
def replay_into_win64(entries, work, home_user, notes):
"""Imports the 32-bit recording's registry into a fresh 64-bit prefix with Wine's 32-bit
regedit and records what landed, Wow6432Node and all."""
prefix = work / "pfx64"
env = wine_env(prefix, "win64", "")
print("== making a fresh win64 prefix to replay the registry into", flush=True)
result = run(["wineboot", "-u"], env=env, timeout=600)
wait_wine(env, 300)
if result.returncode != 0:
notes.append("the 64-bit replay prefix could not be made; registry left in the 32-bit layout")
return entries
before = {"HKLM": parse_reg(prefix / "system.reg"), "HKCU": parse_reg(prefix / "user.reg")}
reg_file = work / "recording.reg"
write_reg(entries, reg_file)
dos_path = "Z:" + str(reg_file.resolve()).replace("/", "\\")
result = run(["wine", "C:\\windows\\syswow64\\regedit.exe", "/S", dos_path], env=env, timeout=600)
wait_wine(env, 300)
subprocess.run(["wineserver", "-k"], env=env)
time.sleep(2)
if result.returncode != 0:
notes.append("32-bit regedit import exit %s: %s" % (result.returncode, result.stderr.strip()[-300:]))
after = {"HKLM": parse_reg(prefix / "system.reg"), "HKCU": parse_reg(prefix / "user.reg")}
replayed = diff_registry(before["HKLM"], after["HKLM"], "HKLM", home_user) + \
diff_registry(before["HKCU"], after["HKCU"], "HKCU", home_user)
# Only what the recording wrote: Wine stirs a few keys of its own in a fresh prefix meanwhile.
wanted = {(e["hive"], e["key"].lower()) for e in entries}
def recorded_key(e):
plain = e["key"].lower().replace("\\wow6432node", "")
return (e["hive"], plain) in wanted or (e["hive"], e["key"].lower()) in wanted
replayed = [e for e in replayed if recorded_key(e)]
wow = sum(1 for e in replayed if "wow6432node" in e["key"].lower())
notes.append("registry replayed through Wine's 32-bit regedit into a 64-bit prefix: %d values in, %d out, %d under Wow6432Node" % (
sum(1 for e in entries if e.get("type") != "key"), sum(1 for e in replayed if e.get("type") != "key"), wow))
print(" replay: %d values in, %d out, %d under Wow6432Node" % (len(entries), len(replayed), wow), flush=True)
return replayed
# ---------------------------------------------------------------- the run
def wine_env(prefix, arch, overrides):
env = dict(os.environ)
env.update({"WINEPREFIX": str(prefix), "WINEARCH": arch, "WINEDEBUG": "-all",
"WINEDLLOVERRIDES": "winemenubuilder.exe=d" + (";" + overrides if overrides else "")})
env.pop("DISPLAY", None)
return env
def wait_wine(env, timeout):
try:
subprocess.run(["wineserver", "-w"], env=env, timeout=timeout)
except subprocess.TimeoutExpired:
subprocess.run(["wineserver", "-k"], env=env)
time.sleep(3)
NESTED_SUFFIXES = (".7z", ".zip", ".cab", ".exe", ".msi", ".rar", ".xz", ".gz", ".tar")
def open_nested(dest, depth=0):
"""Archives inside an extraction (K-Lite carries its filters as 7-Zip payloads) are opened in
place, one folder per archive, two levels deep."""
if depth > 1 or not shutil.which("7z"):
return 0
opened = 0
for path in sorted(p for p in dest.rglob("*") if p.is_file() and p.suffix.lower() in NESTED_SUFFIXES):
if path.name.endswith(".nested") or path.stat().st_size < 1024:
continue
probe = run(["7z", "l", "-ba", str(path)], timeout=300)
if probe.returncode != 0 or not probe.stdout.strip():
continue
folder = path.with_name(path.name + ".nested")
folder.mkdir(exist_ok=True)
run(["7z", "x", "-y", "-bd", "-bso0", "-bsp0", "-o" + str(folder), str(path)], timeout=1800)
if any(folder.rglob("*")):
opened += 1 + open_nested(folder, depth + 1)
else:
shutil.rmtree(folder, ignore_errors=True)
return opened
def extract_installer(installer, dest, notes):
"""Opens the installer with innoextract or 7-Zip; returns {sha256: relative path} of what came out."""
found = {}
tried = []
if shutil.which("innoextract"):
tried.append("innoextract")
listing = run(["innoextract", "-l", str(installer)], timeout=600)
said = (listing.stderr + listing.stdout).strip().splitlines()
warnings = [line for line in said if "warning" in line.lower() or "error" in line.lower()]
print(" innoextract: %d listed, %s" % (sum(1 for line in said if line.startswith(" - ")), "; ".join(warnings[:3]) or "no warnings"), flush=True)
if warnings:
notes.append("innoextract: " + "; ".join(warnings[:3]))
result = run(["innoextract", "-q", "-m", "-d", str(dest), str(installer)], timeout=1800)
if result.returncode != 0 or not any(dest.rglob("*")):
notes.append("innoextract extracted nothing (exit %s)" % result.returncode)
shutil.rmtree(dest, ignore_errors=True)
dest.mkdir(parents=True, exist_ok=True)
if not any(dest.rglob("*")) and shutil.which("7z"):
tried.append("7z")
result = run(["7z", "x", "-y", "-bd", "-bso0", "-bsp0", "-o" + str(dest), str(installer)], timeout=1800)
if not any(dest.rglob("*")):
notes.append("7-Zip extracted nothing (exit %s)" % result.returncode)
top = sum(1 for p in dest.rglob("*") if p.is_file())
nested = open_nested(dest)
for path in dest.rglob("*"):
if path.is_file():
found.setdefault(sha256_of(path), path.relative_to(dest).as_posix())
print(" extraction: %d files at the top, %d nested archives opened, %d files in all" % (top, nested, len(found)), flush=True)
notes.append("extraction: %d files, %d nested archives opened (%s)" % (len(found), nested, ", ".join(tried) or "no extractor"))
return found, tried
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--component", required=True)
parser.add_argument("--installer", help="the installer file (single-installer recipes)")
parser.add_argument("--installers-dir", help="folder holding every recipe's installer assets by name")
parser.add_argument("--out", required=True)
parser.add_argument("--recipes", default=str(Path(__file__).with_name("recipes.json")))
parser.add_argument("--extract", help="where the installer's own extraction goes (default: a temp dir)")
args = parser.parse_args()
recipes = json.loads(Path(args.recipes).read_text("utf-8"))
recipe = recipes.get(args.component)
if not recipe:
print("no recipe for %s" % args.component, file=sys.stderr)
return 2
out = Path(args.out)
out.mkdir(parents=True, exist_ok=True)
arch = recipe.get("arch", "win64")
timeout = int(recipe.get("timeout", 600))
notes = []
assets = Path(args.installers_dir).resolve() if args.installers_dir else None
def runs_of(name):
"""A recipe's installers as (path, args) pairs, from --installers-dir or --installer."""
r = recipes[name]
items = r.get("installers") or [{"asset": r.get("installer"), "args": r.get("args", [])}]
found = []
for item in items:
path = assets / item["asset"] if assets else Path(args.installer).resolve()
if not path.is_file():
raise SystemExit("installer missing: %s" % path)
found.append((path, list(item.get("args", []))))
return found
def run_installer(path, run_args, limit, label):
print("== running %s %s" % (path.name, " ".join(run_args)), flush=True)
cmd = ["xvfb-run", "-a", "-s", "-screen 0 1280x800x24", "wine", str(path)] + run_args
started_at = time.time()
try:
result = run(cmd, env=env, timeout=limit)
exit_code = result.returncode
if result.stderr.strip():
notes.append("%s stderr: %s" % (label, result.stderr.strip()[-400:]))
except subprocess.TimeoutExpired:
exit_code = "timeout"
notes.append("%s was still running after %d s and was killed" % (label, limit))
wait_wine(env, 180)
subprocess.run(["wineserver", "-k"], env=env)
time.sleep(2)
took = int(time.time() - started_at)
print("== %s exit %s after %d s" % (label, exit_code, took), flush=True)
if exit_code != 0:
# What the installer itself said: Microsoft's setups leave dd_*.txt / *.log in Temp.
logs = []
for pattern in ("users/*/Temp/*.txt", "users/*/Temp/*.log", "users/*/Temp/*/*.txt", "users/*/Temp/*/*.log",
"windows/temp/*.txt", "windows/temp/*.log", "windows/temp/*/*.log", "*/*.log", "*/*.txt"):
for log in drive_c.glob(pattern):
try:
if log.stat().st_mtime >= started_at - 5 and log.stat().st_size > 0:
logs.append(log)
except OSError:
pass
logs = sorted(set(logs), key=lambda l: -l.stat().st_size)[:3]
for log in logs:
try:
text = log.read_bytes().decode("utf-16", errors="replace") if log.read_bytes()[:2] in (b"\xff\xfe", b"\xfe\xff") else log.read_text("utf-8", errors="replace")
except OSError:
continue
tail = "\n".join(line for line in text.splitlines() if line.strip())[-1500:]
print(" -- %s (last lines) --\n%s" % (log.relative_to(drive_c), tail), flush=True)
notes.append("%s log %s: %s" % (label, log.relative_to(drive_c).as_posix(), tail[-600:]))
# The lines that name the problem, wherever they are in the log.
telling = [line.strip() for line in text.splitlines()
if re.search(r"block|gencomp|return value 3|error|fail|not (met|found|supported)|requires|missing", line, re.IGNORECASE)
and "does not match requested set" not in line][:25]
if telling:
print(" -- %s (telling lines) --\n%s" % (log.relative_to(drive_c), "\n".join(telling)), flush=True)
return exit_code, took
runs = runs_of(args.component)
installer = runs[0][0]
work = Path(tempfile.mkdtemp(prefix="snapshot-"))
prefix = work / "pfx"
env = wine_env(prefix, arch, recipe.get("dll_overrides", ""))
print("== making a fresh %s prefix" % arch, flush=True)
result = run(["wineboot", "-u"], env=env, timeout=600)
wait_wine(env, 300)
if result.returncode != 0:
print(result.stderr[-800:], file=sys.stderr)
return 1
wine_version = run(["wine", "--version"], env=env).stdout.strip()
drive_c = prefix / "drive_c"
home_user = next((p.name for p in (drive_c / "users").iterdir() if p.is_dir() and p.name.lower() not in ("public", "default")), "")
# The Windows version the installer wants to see (MDAC refuses anything after 2000), set before
# the first snapshot so it is not recorded; prerequisites likewise, so a service pack records its delta.
winver = recipe.get("winver")
if winver:
# winecfg writes the whole set: the version APIs and the registry keys an installer reads
# directly (CurrentVersion, CSDVersion, Control\Windows\CSDVersion). .NET 3.5's "Windows XP
# Service Pack 2" check reads the latter, which the HKCU\Software\Wine\Version shortcut
# never writes.
result = run(["winecfg", "-v", winver], env=env, timeout=300)
wait_wine(env, 120)
if result.returncode != 0:
run(["wine", "reg", "add", "HKCU\\Software\\Wine", "/v", "Version", "/d", winver, "/f"], env=env, timeout=300)
wait_wine(env, 120)
notes.append("recorded with the Windows version set to %s (winecfg)" % winver)
for key in recipe.get("pre_reg_delete", []):
run(["wine", "reg", "delete", key, "/f"], env=env, timeout=300)
wait_wine(env, 120)
notes.append("deleted %s before recording" % key)
for prerequisite in recipe.get("after", []):
for path, run_args in runs_of(prerequisite):
run_installer(path, run_args, int(recipes[prerequisite].get("timeout", 600)), "prerequisite " + prerequisite)
notes.append("recorded on top of %s" % prerequisite)
print("== noting the prefix before", flush=True)
files_before = snapshot_files(drive_c)
reg_before = {"HKLM": parse_reg(prefix / "system.reg"), "HKCU": parse_reg(prefix / "user.reg")}
status, elapsed = None, 0
if recipe.get("extract_to"):
# No silent mode (dirac's NSIS wizard): unpack the installer where its wizard would have put
# the files and register the filters it would have registered, with Wine's own regsvr32.
target = drive_c / recipe["extract_to"]
target.mkdir(parents=True, exist_ok=True)
started_at = time.time()
result = run(["7z", "x", "-y", "-bso0", "-bsp0", "-o" + str(target), str(installer)], timeout=600)
for litter in ("$PLUGINSDIR", "$TEMP", "$R0"):
shutil.rmtree(target / litter, ignore_errors=True)
status = result.returncode
notes.append("unpacked into %s instead of running the wizard (no silent mode)" % recipe["extract_to"])
for rel in recipe.get("register", []):
dos = "C:\\" + rel.replace("/", "\\")
print("== regsvr32 %s" % dos, flush=True)
result = run(["xvfb-run", "-a", "-s", "-screen 0 1280x800x24", "wine", "regsvr32", "/s", dos], env=env, timeout=300)
wait_wine(env, 120)
notes.append("regsvr32 %s exit %s" % (rel, result.returncode))
if result.returncode != 0 and status == 0:
status = result.returncode
subprocess.run(["wineserver", "-k"], env=env)
time.sleep(2)
elapsed = int(time.time() - started_at)
print("== %s unpacked and registered, exit %s after %d s" % (installer.name, status, elapsed), flush=True)
else:
for path, run_args in runs:
exit_code, took = run_installer(path, run_args, timeout, path.name)
elapsed += took
status = exit_code if status in (None, 0) else status
print("== noting the prefix after", flush=True)
files_after = snapshot_files(drive_c)
reg_after = {"HKLM": parse_reg(prefix / "system.reg"), "HKCU": parse_reg(prefix / "user.reg")}
placed = {rel: info for rel, info in files_after.items() if files_before.get(rel) != info}
removed = sorted(rel for rel in files_before if rel not in files_after)
registry = diff_registry(reg_before["HKLM"], reg_after["HKLM"], "HKLM", home_user) + \
diff_registry(reg_before["HKCU"], reg_after["HKCU"], "HKCU", home_user)
# Where each recorded file is read from in the recording prefix (differs from its recorded
# path once a 32-bit recording is moved into the 64-bit layout).
source_of = {rel: rel for rel in placed}
layout = arch
if arch == "win32" and recipe.get("remap", True) and (placed or registry):
moved = {}
for rel, info in placed.items():
target = remap_file(rel)
moved[target] = info
source_of[target] = rel
shifted = sum(1 for rel in placed if remap_file(rel) != rel)
placed = moved
removed = [remap_file(rel) for rel in removed]
# The strings the installer wrote point where it put the files; on 64-bit Windows a 32-bit
# installer resolves "Program Files" to "Program Files (x86)" and writes that. (system32
# may stay: the file-system redirector sends a 32-bit reader to syswow64 by itself.)
program_files = re.compile(r"([A-Za-z]:\\)Program Files\\(?!\(x86\))", re.IGNORECASE)
rewritten = 0
for entry in registry:
data = entry.get("data")
if isinstance(data, str):
new = program_files.sub(r"\1Program Files (x86)\\", data)
elif isinstance(data, list):
new = [program_files.sub(r"\1Program Files (x86)\\", item) for item in data]
else:
continue
if new != data:
entry["data"] = new
rewritten += 1
if rewritten:
notes.append("%d registry strings now say Program Files (x86)" % rewritten)
registry = replay_into_win64(registry, work, home_user, notes)
notes.append("32-bit recording laid out as on 64-bit Windows: %d of %d files moved (syswow64, Program Files (x86))" % (shifted, len(placed)))
layout = "win64"
print("== opening the installer to find the placed files inside it", flush=True)
extract_dir = Path(args.extract) if args.extract else work / "x"
extract_dir.mkdir(parents=True, exist_ok=True)
inside, tried = {}, []
for index, (path, _) in enumerate(runs):
folder = extract_dir / ("installer%d" % index) if len(runs) > 1 else extract_dir
folder.mkdir(parents=True, exist_ok=True)
found, used = extract_installer(path, folder, notes)
for digest, rel in found.items():
inside.setdefault(digest, (("installer%d/" % index) if len(runs) > 1 else "") + rel)
tried = tried or used
# When the installer cannot be opened (Inno Setup newer than innoextract reads, K-Lite) and
# the recipe says its contents may be redistributed, the placed files travel as an archive
# beside the recording instead of being pulled from the installer on the device.
host_files = bool(recipe.get("host_files")) and bool(placed)
files_archive = None
if not placed:
notes.append("the installer placed no files (exit %s): nothing to lay out" % status)
if host_files:
archive = out / ("%s.files.tar.xz" % args.component)
with tempfile.TemporaryDirectory(prefix="files-") as staging:
for rel in placed:
target = Path(staging) / "drive_c" / rel
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(drive_c / source_of[rel], target)
run(["tar", "-C", staging, "-cJf", str(archive), "drive_c"], timeout=1800, check=True)
files_archive = {"name": archive.name, "size": archive.stat().st_size, "sha256": sha256_of(archive),
"licence": recipe.get("licence", "")}
print(" files archive: %s (%d bytes)" % (archive.name, files_archive["size"]), flush=True)
files, inline_total, missing = [], 0, 0
for rel in sorted(placed):
size, digest = placed[rel]
entry = {"path": rel, "size": size, "sha256": digest}
if host_files:
entry["archived"] = True
elif digest in inside:
entry["source"] = inside[digest]
elif size <= INLINE_FILE_LIMIT and inline_total + size <= INLINE_TOTAL_LIMIT:
entry["data"] = base64.b64encode((drive_c / source_of[rel]).read_bytes()).decode("ascii")
inline_total += size
else:
entry["missing"] = True
missing += 1
files.append(entry)
snapshot = {
"schema": 1,
"component": args.component,
"recorded": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"wine": wine_version, "arch": arch, "layout": layout,
"installer": {"name": installer.name, "size": installer.stat().st_size, "sha256": sha256_of(installer),
"args": runs[0][1], "exit": status, "seconds": elapsed, "extractors": tried},
"installers": [{"name": path.name, "size": path.stat().st_size, "sha256": sha256_of(path), "args": run_args} for path, run_args in runs],
"after": recipe.get("after", []), "winver": winver,
"stats": {"files": len(files), "from_installer": sum(1 for f in files if "source" in f),
"inline": sum(1 for f in files if "data" in f), "archived": sum(1 for f in files if f.get("archived")),
"missing": missing, "removed": len(removed), "registry": len(registry)},
"files_archive": files_archive,
"files": files, "removed": removed, "registry": registry, "notes": notes,
}
(out / ("%s.snapshot.json" % args.component)).write_text(json.dumps(snapshot, indent=1), "utf-8")
by_hive = {}
for value in registry:
by_hive.setdefault(value["hive"], set()).add(value["key"].split("\\")[0] + "\\" + value["key"].split("\\")[1] if "\\" in value["key"] else value["key"])
summary = ["# %s" % args.component, "",
"Recorded %s on %s (%s prefix%s). Installer `%s`, exit %s after %d s." % (
snapshot["recorded"], wine_version, arch, ", laid out for win64" if layout != arch else "", installer.name, status, elapsed), "",
"| | |", "|---|---|",
"| files placed | %d |" % len(files),
"| of which found inside the installer | %d |" % snapshot["stats"]["from_installer"],
"| carried in the files archive beside this recording | %d |" % snapshot["stats"]["archived"],
"| carried inline (generated, small) | %d |" % snapshot["stats"]["inline"],
"| missing (generated, large) | %d |" % missing,
"| files removed | %d |" % len(removed),
"| registry values | %d |" % len(registry), "",
"## Where the files went", ""]
folders = {}
for entry in files:
top = "/".join(entry["path"].split("/")[:3])
folders[top] = folders.get(top, 0) + 1
summary += ["- `%s` · %d" % (top, n) for top, n in sorted(folders.items(), key=lambda item: -item[1])[:25]]
summary += ["", "## Registry keys touched (top two levels)", ""]
for hive, keys in sorted(by_hive.items()):
for key in sorted(keys)[:40]:
summary.append("- %s\\%s" % (hive, key))
if missing:
summary += ["", "## Missing (not inside the installer, too large to carry)", ""]
summary += ["- `%s` (%d bytes)" % (f["path"], f["size"]) for f in files if f.get("missing")][:50]
if notes:
summary += ["", "## Notes", ""] + ["- " + n for n in notes]
(out / ("%s.summary.md" % args.component)).write_text("\n".join(summary) + "\n", "utf-8")
print("\n".join(summary[:12]), flush=True)
shutil.rmtree(work, ignore_errors=True)
return 0
if __name__ == "__main__":
sys.exit(main())