Add Docker support and update README (#13)

* add compose.yaml

* add Dockerfile

* update README

* Dockerfile: uncomment CMD instruction

* add .dockerignore

* Dockerfile: use scratch image for second stage

* Dockerfile: use newer OpenSSL version (3.0.9)

* Dockerfile: use Alpine-based Rust image to skip building OpenSSL

* Dockerfile: expose to localhost only by default

* Dockerfile: allow building for different architectures (AMD64 and ARM64)

Dockerfile now detects the architecture being used during the build process and templates in the correct Rust target architecture

* compose.yaml: include command line

* compose.yaml: listen on 127.0.0.1 only

* compose.yaml: remove healthcheck due to using scratch image

* README.md: restore old table format

* rename compose.yaml to docker-compose.yml

* docker-compose.yml: include version line

* update .dockerignore

* README.md: update Docker instructions to expose only on localhost

* add workflow to build and push container images to quay.io

* docker-compose.yml: harden configuration

* docker workflow: add paths-ignore section

* Dockerfile: let Rustup handle architecture detection

More flexible as the build process will now automatically adapt to whatever architecture the container is being built on, without needing to explicitly list out each supported architecture

* Docker: further security hardening

- Run as a non-privileged user within the scratch container
- Add security_opt: - no-new-privileges:true to docker-compose.yml

* Dockerfile: Switch to Debian-based images and simplify build command

- rust:1.80-alpine replaced with rust:1.80 for the builder stage
- alpine:3.20 replaced with debian:12.6-slim for the user-stage
- Build command simplified to use default target architecture

* Cargo.toml: correct note on optimisations

* docker-compose.yml: use quay.io image by default

* rename docker-compose.yml to docker-compose.yaml

* compose: build image from local repo by default

* Revert "Dockerfile: Switch to Debian-based images and simplify build command"

This reverts commit ff9a378564.

Reasons for reverting:
1. Compiling via musl is necessary to statically link dependencies and create a truly standalone Rust binary. [1]
2. Alpine-based Rust images are required for the build stage because such systems support dynamic linking, which is also needed for statically-linked binaries. [2]
3. Determining the target architecture and templating the correct value for the --target flag is necessary for the statically-linked binary to be built correctly. [2]

[1]: https://doc.rust-lang.org/1.13.0/book/advanced-linking.html#linux
[2]: https://github.com/rust-lang/rust/issues/40174#issuecomment-538791091
This commit is contained in:
perennial 2024-08-14 20:58:44 +10:00 committed by GitHub
parent f7cbcbddfa
commit 4d3cff6a16
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 270 additions and 22 deletions

13
.dockerignore Normal file
View File

@ -0,0 +1,13 @@
# Rust build artifacts
/target
# Git files
.git
.gitignore
# Docker Compose file
docker-compose.yaml
# Documentation
README.md
LICENSE

View File

@ -0,0 +1,72 @@
name: Build and Push Docker Image
# Define when this workflow will run
on:
push:
branches:
- master # Trigger on pushes to master branch
tags:
- '[0-9]+.[0-9]+.[0-9]+' # Trigger on semantic version tags
paths-ignore:
- 'Cargo.lock'
- 'LICENSE'
- 'README.md'
- 'docker-compose.yml'
workflow_dispatch: # Allow manual triggering of the workflow
# Define environment variables used throughout the workflow
env:
REGISTRY: quay.io
IMAGE_NAME: invidious/inv-sig-helper
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
# Step 1: Check out the repository code
- name: Checkout code
uses: actions/checkout@v3
# Step 2: Set up QEMU for multi-architecture builds
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
# Step 3: Set up Docker Buildx for enhanced build capabilities
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
# Step 4: Authenticate with Quay.io registry
- name: Log in to Quay.io
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_PASSWORD }}
# Step 5: Extract metadata for Docker image tagging and labeling
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# Define tagging strategy
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=sha,prefix={{branch}}-
# Define labels
labels: |
quay.expires-after=12w
# Step 6: Build and push the Docker image
- name: Build and push Docker image
uses: docker/build-push-action@v4
with:
context: .
push: true
platforms: linux/amd64,linux/arm64 # Build for multiple architectures
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

View File

@ -14,3 +14,11 @@ lazy-regex = "3.1.0"
tub = "0.3.7" tub = "0.3.7"
tokio-util = { version = "0.7.10", features=["futures-io", "futures-util", "codec"]} tokio-util = { version = "0.7.10", features=["futures-io", "futures-util", "codec"]}
futures = "0.3.30" futures = "0.3.30"
# Compilation optimizations for release builds
# Increases compile time but typically produces a faster and smaller binary. Suitable for final releases but not for debug builds.
[profile.release]
lto = true
opt-level = 3
codegen-units = 1
panic = 'abort'

55
Dockerfile Normal file
View File

@ -0,0 +1,55 @@
# Use the official Alpine-based Rust image as a parent image
FROM rust:1.80-alpine AS builder
# Set the working directory in the container
WORKDIR /usr/src/app
# Install build dependencies
RUN apk add --no-cache \
musl-dev \
openssl-dev \
openssl-libs-static \
pkgconfig \
patch
# Set environment variables for static linking
ENV OPENSSL_STATIC=yes
ENV OPENSSL_DIR=/usr
# Copy the current directory contents into the container
COPY . .
# Determine the target architecture and build the application
RUN RUST_TARGET=$(rustc -vV | sed -n 's/host: //p') && \
rustup target add $RUST_TARGET && \
RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target $RUST_TARGET
# Stage for creating the non-privileged user
FROM alpine:3.20 AS user-stage
RUN adduser -u 10001 -S appuser
# Stage for a smaller final image
FROM scratch
# Copy necessary files from the builder stage, using the correct architecture path
COPY --from=builder /usr/src/app/target/*/release/inv_sig_helper_rust /app/inv_sig_helper_rust
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
# Copy passwd file for the non-privileged user from the user-stage
COPY --from=user-stage /etc/passwd /etc/passwd
# Set the working directory
WORKDIR /app
# Expose port 12999
EXPOSE 12999
# Switch to non-privileged user
USER appuser
# Set the entrypoint to the binary name
ENTRYPOINT ["/app/inv_sig_helper_rust"]
# Set default arguments in CMD
CMD ["--tcp", "127.0.0.1:12999"]

125
README.md
View File

@ -1,8 +1,85 @@
# Protocol Format # inv_sig_helper
`inv_sig_helper` is a Rust service that decrypts YouTube signatures and manages player information. It offers a TCP/Unix socket interface for signature decryption and related operations.
## Features
- Decrypt YouTube `n` and `s` signatures
- Fetch and update YouTube player data
- Provide signature timestamps and player status
- Efficient signature decryption with multi-threaded JavaScript execution
## Building and Running
### Prerequisites
- Rust 1.77 or later
- Cargo
- Patch
- openssl-devel
### Building
1. Clone the repository and navigate to the project directory:
```
git clone https://github.com/iv-org/inv_sig_helper.git
cd inv_sig_helper
```
2. Build the project:
```
cargo build --release
```
### Running
The service can run in Unix socket mode (default) or TCP mode:
1. Unix socket mode:
```
./target/release/inv_sig_helper_rust
```
This creates a Unix socket at `/tmp/inv_sig_helper.sock`.
2. TCP mode:
```
./target/release/inv_sig_helper_rust --tcp [IP:PORT]
```
If no IP:PORT is given, it defaults to `127.0.0.1:12999`.
## Docker
A Dockerfile is included for containerized deployment.
1. Build the image:
```
docker build -t inv_sig_helper .
```
2. Run the container:
```
docker run -p 127.0.0.1:12999:12999 inv_sig_helper
```
Or use Docker Compose:
```
docker compose up
```
## Protocol Format
All data-types bigger than 1 byte are stored in network endian (big-endian) unless stated otherwise. All data-types bigger than 1 byte are stored in network endian (big-endian) unless stated otherwise.
## Request Base ### Request Base
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|-----------|--------------|--------------------------------------| |-----------|--------------|--------------------------------------|
|opcode | 1 | The operation code to perform, A list of operations currently supported (and their data) can be found in the **Operations** chapter | |opcode | 1 | The operation code to perform, A list of operations currently supported (and their data) can be found in the **Operations** chapter |
@ -10,7 +87,7 @@ All data-types bigger than 1 byte are stored in network endian (big-endian) unle
The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information. The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.
## Response Base ### Response Base
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------------|--------------|---------------------------------------| |------------|--------------|---------------------------------------|
|request_id | 4 | The ID for the request that this response is meant for | |request_id | 4 | The ID for the request that this response is meant for |
@ -18,80 +95,84 @@ The data afterwards depends on the supplied opcode, Please consult the **Operati
The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information. The data afterwards depends on the supplied opcode, Please consult the **Operations** chapter for more information.
## Operations ### Operations
### `FORCE_UPDATE` (0x00) #### `FORCE_UPDATE` (0x00)
Forces the server to re-fetch the YouTube player, and extract the necessary components from it (`nsig` function code, `sig` function code, signature timestamp). Forces the server to re-fetch the YouTube player, and extract the necessary components from it (`nsig` function code, `sig` function code, signature timestamp).
#### Request ##### Request
*No additional data required* *No additional data required*
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------|--------------|-------------| |------|--------------|-------------|
|status| 2 | The status code of the request: `0xF44F` if successful, `0xFFFF` if no updating is required (YouTube's player ID is equal to the server's current player ID), `0x0000` if an error occurred | |status| 2 | The status code of the request: `0xF44F` if successful, `0xFFFF` if no updating is required (YouTube's player ID is equal to the server's current player ID), `0x0000` if an error occurred |
### `DECRYPT_N_SIGNATURE` (0x01) #### `DECRYPT_N_SIGNATURE` (0x01)
Decrypt a provided `n` signature using the server's current `nsig` function code, and return the result (or an error). Decrypt a provided `n` signature using the server's current `nsig` function code, and return the result (or an error).
#### Request ##### Request
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------|--------------|-------------------------------------| |------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature | |size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature | |string| *`size`* | The encrypted signature |
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------| |------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred | |size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature | |string| *`size`* | The decrypted signature |
### `DECRYPT_SIGNATURE` (0x02) #### `DECRYPT_SIGNATURE` (0x02)
Decrypt a provided `s` signature using the server's current `sig` function code, and return the result (or an error). Decrypt a provided `s` signature using the server's current `sig` function code, and return the result (or an error).
#### Request ##### Request
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------|--------------|-------------------------------------| |------|--------------|-------------------------------------|
|size | 2 | The size of the encrypted signature | |size | 2 | The size of the encrypted signature |
|string| *`size`* | The encrypted signature | |string| *`size`* | The encrypted signature |
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|------|--------------|------------------------------------------------------------------| |------|--------------|------------------------------------------------------------------|
|size | 2 | The size of the decrypted signature, `0x0000` if an error occurred | |size | 2 | The size of the decrypted signature, `0x0000` if an error occurred |
|string| *`size`* | The decrypted signature | |string| *`size`* | The decrypted signature |
### `GET_SIGNATURE_TIMESTAMP` (0x03) #### `GET_SIGNATURE_TIMESTAMP` (0x03)
Get the signature timestamp from the server's current player, and return it in the form of a 64-bit integer. If there's no player, it will return 0 in the `timestamp` (Please check with `PLAYER_STATUS` if the server has a player) Get the signature timestamp from the server's current player, and return it in the form of a 64-bit integer. If there's no player, it will return 0 in the `timestamp` (Please check with `PLAYER_STATUS` if the server has a player)
#### Request ##### Request
No additional data required No additional data required
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|---------|--------------|----------------------------------------------------------| |---------|--------------|----------------------------------------------------------|
|timestamp| 8 | The signature timestamp from the server's current player | |timestamp| 8 | The signature timestamp from the server's current player |
### `PLAYER_STATUS` (0x04) #### `PLAYER_STATUS` (0x04)
Get the server's information about the current player. Get the server's information about the current player.
#### Request ##### Request
No additional data required No additional data required
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|----------|--------------|-------------| |----------|--------------|-------------|
|has_player| 1 | If the server has a player, this variable will be `0xFF`. or else, it will be `0x00`| |has_player| 1 | If the server has a player, this variable will be `0xFF`. or else, it will be `0x00`|
|player_id | 4 | The server's current player ID. If the server has no player, this will always be `0x00000000`| |player_id | 4 | The server's current player ID. If the server has no player, this will always be `0x00000000`|
### `PLAYER_UPDATE_TIMESTAMP` (0x05) #### `PLAYER_UPDATE_TIMESTAMP` (0x05)
Get the time of the last player update, The time is represented as seconds since the last update Get the time of the last player update, The time is represented as seconds since the last update
#### Request ##### Request
No additional data required No additional data required
#### Response ##### Response
| Name | Size (bytes) | Description | | Name | Size (bytes) | Description |
|----------|--------------|-------------| |----------|--------------|-------------|
|timestamp | 8 | Seconds since the last player update | |timestamp | 8 | Seconds since the last player update |
## License
This project is open source under the AGPL-3.0 license.

19
docker-compose.yaml Normal file
View File

@ -0,0 +1,19 @@
version: "3"
services:
inv_sig_helper:
build:
context: .
dockerfile: Dockerfile
# image: quay.io/invidious/inv-sig-helper:latest
command: ["--tcp", "127.0.0.1:12999"]
ports:
- 127.0.0.1:12999:12999
environment:
- RUST_LOG=info
restart: unless-stopped
cap_drop:
- ALL
read_only: true
user: 10001:10001
security_opt:
- no-new-privileges:true