From 104f91e697d8c5c4bb71ce995c2b95ab75d84dae Mon Sep 17 00:00:00 2001 From: Uruk Date: Fri, 23 Feb 2024 17:52:10 +0100 Subject: [PATCH] Refactor jellyfin-packaging - Fix typo in the README - Added -f to rm from fix https://github.com/jellyfin/jellyfin/pull/11008 - Apply fix from https://github.com/jellyfin/jellyfin/pull/10763 - Refactor Docker for better syntax and readability - Update Intel runtime drivers to 24.05.28454.6 - Added possible miss line 159 docker/Dockerfile : "--ffmpeg", "/usr/lib/jellyfin-ffmpeg/ffmpeg" - Removed wget in cleanup --- README.md | 16 ++++++------ debian/conf/jellyfin.init | 5 ++-- debian/docker/Dockerfile | 16 ++++++------ debian/rules | 2 +- docker/Dockerfile | 53 +++++++++++++-------------------------- portable/Dockerfile | 13 ++++++---- 6 files changed, 45 insertions(+), 60 deletions(-) diff --git a/README.md b/README.md index b3b485e..9afaf65 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ Inside this repository are 7 major components: 1. Submodules for the `jellyfin` (as `jellyfin-server`) and `jellyfin-web` repositories. These are dynamic submodules; the `checkout.py` script will check them out to the required `HEAD` on each build, and thus their actual committed value is irrelevant. Nonetheless, they should be bumped occasionally just to avoid excessive checkout times later. -2. Debian/Ubuntu packaging configurations (under `debian`). These will build the 3 Jellyfin packages (`jellyfin` metapackage, `jellyfin-server` core server, and `jellyfin-web` web client) from a single Dockerfile and helper script (`build.sh`) under `debian/docker/`. Future packages (e.g. Vue) may be added here as well if and when they are promoted to a production build alongside the others, following one consistent versioning scheme. +2. Debian/Ubuntu packaging configurations (under `debian`). These will build the 3 Jellyfin packages (`jellyfin` metapackage, `jellyfin-server` core server, and `jellyfin-web` web client) from a single Dockerfile and helper script (`build.sh`) under `debian/docker/`. Future packages (e.g. Vue) may be added here as well as if and when they are promoted to a production build alongside the others, following one consistent versioning scheme. 3. Fedora/CentOS packaging configurations (under `fedora`). These will build the same packages as Debian. This is still a TODO. @@ -69,7 +69,7 @@ Inside this repository are 7 major components: * Unified packaging: all packaging is in this repository (vs. within the `jellyfin-server` and `jellyfin-web` repositories) This helps ensure two things: - 1. There is a single source of truth for packaging. Previously, there were at least 3 sources of truth and this became very confusing to update. + 1. There is a single source of truth for packaging. Previously, there were at least 3 sources of truth, and this became very confusing to update. 2. Packaging can be run and managed independently of actual code, simplifying the release and build process. * GitHub Actions for CI: all builds use the GitHub Actions system instead of Azure DevOps @@ -82,7 +82,7 @@ Inside this repository are 7 major components: * Git Submodules to handle code (vs. cross-repo builds) - This ensures that the code checked out is consistent to both repositories, and allows for the unified builds described below without extra steps to combine. + This ensures that the code checked out is consistent to both repositories and allows for the unified builds described below without extra steps to combine. * Remote manual-only triggers: CI workers are triggered by a remote bot @@ -90,7 +90,7 @@ Inside this repository are 7 major components: ### Debian/Ubuntu Packages -* Unified package build: this entire repo is the "source" and the source package is named "jellyfin". +* Unified packages build: this entire repo is the "source" and the source package is named "jellyfin". This was chosen to simplify the source package system and simplify building. Now, there is only a single "jellyfin" source package rather than 2. There may be more in the future as other repos might be included (e.g. "jellyfin-ffmpeg", "jellyfin-vue", etc.) @@ -104,7 +104,7 @@ Inside this repository are 7 major components: * Ubuntu LTS-only support: non-LTS Ubuntu versions have been dropped - This simplifies our builds as we do not need to then track many 9-month-only releases of Ubuntu, and also reduces the build burden. Users of non-LTS Ubuntu releases can use either the closest Ubuntu LTS version, or use Docker containers instead. + This simplifies our builds as we do not need to then track many 9-month-only releases of Ubuntu, and also reduces the build burden. Users of non-LTS Ubuntu releases can use either the closest Ubuntu LTS version or use Docker containers instead. ### Fedora/CentOS Packages @@ -112,7 +112,7 @@ TODO - these have not yet been implemented. ### Docker -* Single unified Docker build: the entirety of our Docker images are built as one container from one Dockerfile. +* Single unified Docker build: the entirety of our Docker images is built as one container from one Dockerfile. This was chosen to keep our Docker builds as simple as possible without requiring 2 intervening images (as was the case with our previous CI). @@ -132,11 +132,11 @@ TODO - these have not yet been implemented. * Single unified build: the entirety of the output package is built in one container from one Dockerfile - This was chosen to keep the portable builds as simple as possible without requiring complex archive combining (as was the case with our previous CI). + This was chosen to keep the portable builds as simple as possible without requiring complex archives combining (as was the case with our previous CI). * Multiple archive type support (`.tar.gz` vs. `.zip`) - The output archive type (`.tar.gz` or `.zip`) is chosen based on the build target, with Portable providing both for maximum compatibility, Windows providing `.zip`, and Linux and MacOS providing `.tar.gz`. This can be changed later, for example to add more formats (e.g. `.tar.xz`) or change what produces what, without major complications. + The output archive type (`.tar.gz` or `.zip`) is chosen based on the build target, with Portable providing both for maximum compatibility, Windows providing `.zip`, Linux and MacOS provide `.tar.gz`. This can be changed later, for example to add more formats (e.g. `.tar.xz`) or change what produces what, without major complications. * Full architecture support diff --git a/debian/conf/jellyfin.init b/debian/conf/jellyfin.init index 784536d..72cf2e2 100644 --- a/debian/conf/jellyfin.init +++ b/debian/conf/jellyfin.init @@ -19,13 +19,14 @@ fi . /lib/lsb/init-functions +JELLYFIN_HOME="/var/lib/jellyfin/" PIDFILE="/run/jellyfin.pid" case "$1" in start) log_daemon_msg "Starting Jellyfin Media Server" "jellyfin" || true - if start-stop-daemon --start --quiet --oknodo --background --pidfile $PIDFILE --make-pidfile --user $JELLYFIN_USER --chuid $JELLYFIN_USER --exec /usr/bin/jellyfin -- $JELLYFIN_ARGS; then + if start-stop-daemon --start --quiet --oknodo --background --chdir $JELLYFIN_HOME --pidfile $PIDFILE --make-pidfile --user $JELLYFIN_USER --chuid $JELLYFIN_USER --exec /usr/bin/jellyfin -- $JELLYFIN_ARGS; then log_end_msg 0 || true else log_end_msg 1 || true @@ -44,7 +45,7 @@ case "$1" in restart) log_daemon_msg "Restarting Jellyfin Media Server" "jellyfin" || true start-stop-daemon --stop --quiet --oknodo --retry 30 --pidfile $PIDFILE --remove-pidfile - if start-stop-daemon --start --quiet --oknodo --background --pidfile $PIDFILE --make-pidfile --user $JELLYFIN_USER --chuid $JELLYFIN_USER --exec /usr/bin/jellyfin -- $JELLYFIN_ARGS; then + if start-stop-daemon --start --quiet --oknodo --background --chdir $JELLYFIN_HOME --pidfile $PIDFILE --make-pidfile --user $JELLYFIN_USER --chuid $JELLYFIN_USER --exec /usr/bin/jellyfin -- $JELLYFIN_ARGS; then log_end_msg 0 || true else log_end_msg 1 || true diff --git a/debian/docker/Dockerfile b/debian/docker/Dockerfile index 02caa7b..8766278 100644 --- a/debian/docker/Dockerfile +++ b/debian/docker/Dockerfile @@ -32,11 +32,8 @@ RUN apt-get update -y \ && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC \ apt-get install --no-install-recommends -y \ wget debhelper gnupg devscripts build-essential mmv lsb-release \ - libssl*.* liblttng-ust* \ - libfontconfig*-dev libcurl*openssl-dev libfreetype*-dev libssl-dev \ - && apt-get clean autoclean -y \ - && apt-get autoremove -y \ - && rm -rf /var/lib/apt/lists/* + libssl*.* liblttng-ust* libfontconfig*-dev libcurl*openssl-dev \ + libfreetype*-dev libssl-dev # Prepare the cross-toolchain RUN if test "${PACKAGE_ARCH}" != "$( dpkg --print-architecture )"; then \ @@ -67,9 +64,6 @@ RUN if test "${PACKAGE_ARCH}" != "$( dpkg --print-architecture )"; then \ libc6-dev:${PACKAGE_ARCH} linux-libc-dev:${PACKAGE_ARCH} libgcc1:${PACKAGE_ARCH} libstdc++-${GCC_VERSION}-dev:${PACKAGE_ARCH} \ libfontconfig*-dev:${PACKAGE_ARCH} libcurl*openssl-dev:${PACKAGE_ARCH} libfreetype*-dev:${PACKAGE_ARCH} libssl-dev:${PACKAGE_ARCH} \ libssl[13].*:${PACKAGE_ARCH} liblttng-ust*:${PACKAGE_ARCH} \ - && apt-get clean autoclean -y \ - && apt-get autoremove -y \ - && rm -rf /var/lib/apt/lists/* \ ; fi # Prepare dotnet SDK @@ -86,6 +80,12 @@ RUN wget https://deb.nodesource.com/setup_${NODEJS_VERSION}.x -O nodejs-install. && apt-get install -y \ nodejs +# Clean up dependencies +RUN apt-get remove wget -y \ + && apt-get clean autoclean -y \ + && apt-get autoremove -y \ + && rm -rf /var/cache/apt/archives/* /var/lib/apt/lists/* + # Link to build script RUN ln -sf ${SOURCE_DIR}/debian/docker/build.sh /build.sh diff --git a/debian/rules b/debian/rules index 678c4d8..e8a07bb 100755 --- a/debian/rules +++ b/debian/rules @@ -42,7 +42,7 @@ override_dh_auto_build: cd jellyfin-server && dotnet publish -maxcpucount:1 --configuration $(CONFIG) \ --output='$(CURDIR)/build/usr/lib/jellyfin/bin' --self-contained --runtime $(DOTNETRUNTIME) \ -p:DebugSymbols=false -p:DebugType=none Jellyfin.Server - rm $(CURDIR)/build/usr/lib/jellyfin/bin/libcoreclrtraceptprovider.so + rm -f $(CURDIR)/build/usr/lib/jellyfin/bin/libcoreclrtraceptprovider.so cd jellyfin-web && npm ci --no-audit --unsafe-perm && \ npm run build:production && mv $(CURDIR)/jellyfin-web/dist $(CURDIR)/build/web diff --git a/docker/Dockerfile b/docker/Dockerfile index a8ab0a1..fa0054a 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -9,10 +9,10 @@ ARG OS_VERSION=bookworm ARG FFMPEG_PACKAGE=jellyfin-ffmpeg6 # https://github.com/intel/compute-runtime/releases -ARG GMMLIB_VERSION=22.3.0 -ARG IGC_VERSION=1.0.14828.8 -ARG NEO_VERSION=23.30.26918.9 -ARG LEVEL_ZERO_VERSION=1.3.26918.9 +ARG GMMLIB_VERSION=22.3.11 +ARG IGC_VERSION=1.0.15985.7 +ARG NEO_VERSION=24.05.28454.6 +ARG LEVEL_ZERO_VERSION=1.3.28454.6 # Debian architecture (amd64, arm64, armhf), set by build script ARG PACKAGE_ARCH @@ -38,19 +38,9 @@ ARG JELLYFIN_VERSION ENV JELLYFIN_VERSION=${JELLYFIN_VERSION} RUN apk add \ - autoconf \ - g++ \ - make \ - libpng-dev \ - gifsicle \ - alpine-sdk \ - automake \ - libtool \ - make \ - gcc \ - musl-dev \ - nasm \ - python3 + autoconf g++ make libpng-dev \ + gifsicle alpine-sdk automake \ + libtool gcc musl-dev nasm python3 WORKDIR ${SOURCE_DIR} COPY jellyfin-web . @@ -122,26 +112,17 @@ ENV NVIDIA_VISIBLE_DEVICES="all" ENV NVIDIA_DRIVER_CAPABILITIES="compute,video,utility" # Install dependencies: -# curl: healthcheck RUN apt-get update \ && apt-get install --no-install-recommends --no-install-suggests --yes \ - ca-certificates \ - gnupg \ - curl \ - wget \ - apt-transport-https \ + ca-certificates gnupg curl \ && curl -fsSL https://repo.jellyfin.org/jellyfin_team.gpg.key \ | gpg --dearmor -o /etc/apt/trusted.gpg.d/debian-jellyfin.gpg \ && echo "deb [arch=${PACKAGE_ARCH}] https://repo.jellyfin.org/debian ${OS_VERSION} main" > /etc/apt/sources.list.d/jellyfin.list \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests --yes \ - ${FFMPEG_PACKAGE} \ - openssl \ - locales \ - libfontconfig1 \ - libfreetype6 \ + ${FFMPEG_PACKAGE} openssl locales libfontconfig1 libfreetype6 \ && sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen \ - && apt-get remove gnupg apt-transport-https --yes \ + && apt-get remove gnupg --yes \ && apt-get clean autoclean --yes \ && apt-get autoremove --yes \ && rm -rf /var/cache/apt/archives* /var/lib/apt/lists/* @@ -153,16 +134,15 @@ RUN apt-get update \ RUN if [[ ${PACKAGE_ARCH} == "amd64" ]]; then \ mkdir intel-compute-runtime \ && pushd intel-compute-runtime \ - && wget https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/libigdgmm12_${GMMLIB_VERSION}_amd64.deb \ - && wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-${IGC_VERSION}/intel-igc-core_${IGC_VERSION}_amd64.deb \ - && wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-${IGC_VERSION}/intel-igc-opencl_${IGC_VERSION}_amd64.deb \ - && wget https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/intel-opencl-icd_${NEO_VERSION}_amd64.deb \ - && wget https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/intel-level-zero-gpu_${LEVEL_ZERO_VERSION}_amd64.deb \ + && curl -LO https://github.com/intel/intel-graphics-compiler/releases/download/igc-${IGC_VERSION}/intel-igc-core_${IGC_VERSION}_amd64.deb \ + -LO https://github.com/intel/intel-graphics-compiler/releases/download/igc-${IGC_VERSION}/intel-igc-opencl_${IGC_VERSION}_amd64.deb \ + -LO https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/intel-opencl-icd_${NEO_VERSION}_amd64.deb \ + -LO https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/intel-level-zero-gpu_${LEVEL_ZERO_VERSION}_amd64.deb \ + -LO https://github.com/intel/compute-runtime/releases/download/${NEO_VERSION}/libigdgmm12_${GMMLIB_VERSION}_amd64.deb \ && dpkg -i *.deb \ && popd \ && rm -rf intel-compute-runtime \ ; fi \ - && apt-get remove wget --yes \ && apt-get clean autoclean --yes \ && apt-get autoremove --yes \ && rm -rf /var/cache/apt/archives* /var/lib/apt/lists/* @@ -175,7 +155,8 @@ COPY --from=web /web /jellyfin/jellyfin-web EXPOSE 8096 VOLUME ${JELLYFIN_DATA_DIR} ${JELLYFIN_CACHE_DIR} -ENTRYPOINT [ "/jellyfin/jellyfin" ] +ENTRYPOINT [ "/jellyfin/jellyfin", \ + "--ffmpeg", "/usr/lib/jellyfin-ffmpeg/ffmpeg" ] HEALTHCHECK --interval=30s --timeout=30s --start-period=10s --retries=3 \ CMD curl -Lk -fsS "${HEALTHCHECK_URL}" || exit 1 diff --git a/portable/Dockerfile b/portable/Dockerfile index afe33d7..73856f1 100644 --- a/portable/Dockerfile +++ b/portable/Dockerfile @@ -26,11 +26,8 @@ RUN apt-get update -y \ && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC \ apt-get install --no-install-recommends -y \ wget debhelper gnupg devscripts build-essential mmv lsb-release zip \ - libssl*.* liblttng-ust* \ - libfontconfig*-dev libcurl*openssl-dev libfreetype*-dev libssl-dev \ - && apt-get clean autoclean -y \ - && apt-get autoremove -y \ - && rm -rf /var/cache/apt/archives/* /var/lib/apt/lists/* + libssl*.* liblttng-ust* libssl-dev \ + libfontconfig*-dev libcurl*openssl-dev libfreetype*-dev # Prepare dotnet SDK RUN wget https://packages.microsoft.com/config/${PACKAGE_TYPE}/${PACKAGE_VERSION}/packages-microsoft-prod.deb -O packages-microsoft-prod.deb \ @@ -46,6 +43,12 @@ RUN wget https://deb.nodesource.com/setup_${NODEJS_VERSION}.x -O nodejs-install. && apt-get install -y \ nodejs +# Clean up dependencies +RUN apt-get remove wget -y \ + && apt-get clean autoclean -y \ + && apt-get autoremove -y \ + && rm -rf /var/cache/apt/archives/* /var/lib/apt/lists/* + # Link to build script RUN ln -sf ${SOURCE_DIR}/portable/build.sh /build.sh