crowdsecurity/crowdsec
1
0
Fork 0
mirror of https://github.com/crowdsecurity/crowdsec.git synced 2026-10-12 02:12:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

CI hardening (#4546)

Browse Source 
* ci: set least-privilege permissions on workflows

Add an explicit top-level permissions: block to every workflow that was
relying on the default GITHUB_TOKEN scope, granting only what each one
needs (contents: read by default; packages: write for the ghcr.io pushes;
actions: write for the cache cleanup).

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci: disable credential persistence on checkout

Set persist-credentials: false on every actions/checkout step so the
GITHUB_TOKEN is not written to .git/config for the job's lifetime. None
of these jobs push via git, so nothing relied on the persisted token.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci: pass untrusted values via env instead of inline interpolation

In version.yml and cache-cleanup.yaml, move workflow-dispatch- and
pull_request-derived values out of the inline run: script and into env:
variables, removing the shell-injection surface.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci: verify protoc and javy download checksums

The go generate check downloaded protoc and javy release binaries over
curl with no integrity check. Pin the sha256 of each asset and verify it
before use (job aborts on mismatch via set -e), and harden the curl
invocation (--fail, --proto '=https', --tlsv1.2).

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci: use native gh cache instead of third-party extension

Cache cleanup installed the actions/gh-actions-cache extension at runtime
(an unpinned, unverified download). gh now manages Actions caches
natively, so replace the extension + delete loop with a single
'gh cache delete --all --ref ...'. This also drops the now-unnecessary
repo checkout (gh reads the repo from GH_REPO) and narrows permissions to
just actions: write.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci: inline coverage badge update instead of third-party action

Replace schneegans/dynamic-badges-action with an inline gh api PATCH to
the badge gist. The action was just wrapping a GitHub Gist update with
shields.io endpoint JSON, so this removes a single-maintainer dependency
in favor of the GitHub API we already trust, and updates both badge
files in one call.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* test(bats): fetch hub index from a git clone to dodge CDN rate limiting

The bats fixture builds fetch the hub index from cdn-hub.crowdsec.net,
which rate-limits shared CI runner IPs and intermittently 403s, failing
'cscli hub update'. The committed .index.json on each hub branch already
embeds every item's content, so 'collections install' needs no further
downloads once the index is present.

Add a TEST_HUB_CLONE switch to config-local: when set, fetch the index
from a shallow git clone of the hub at the pinned branch instead of the
CDN (default behavior is unchanged, so local dev still uses the CDN). All
four bats CI jobs set it. On a failed index fetch (CDN or clone) log the
runner's public IP to help diagnose future rate limiting.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

* ci(docker): use a GitHub environment for docker publish secrets

Set the reusable publish-docker workflow's push job to a configurable
environment (default: docker) so DockerHub credentials are sourced from
environment secrets. Drop the now-redundant secrets forwarding from the
master and release callers.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 <[email protected]>
This commit is contained in:
blotus
2026-07-08 10:03:30 +02:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 112cc23e6c
commit 7364ef838e
18 changed files with 140 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/bats-hub.yml
+22 -20
View File
@@ -8,6 +8,9 @@ on:
GIST_BADGES_ID:
required: true
permissions:
contents: read
jobs:
build:
@@ -26,6 +29,7 @@ jobs:
with:
fetch-depth: 0
submodules: true
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
@@ -39,6 +43,9 @@ jobs:
sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev
- name: "Build crowdsec and fixture"
# fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
env:
TEST_HUB_CLONE: "true"
run: make bats-clean bats-build bats-fixture BUILD_STATIC=1
- name: "Run hub tests"
@@ -52,24 +59,19 @@ jobs:
- name: "Collect hub coverage"
run: ./test/bin/collect-hub-coverage ./hub >> $GITHUB_ENV
- name: "Create Parsers badge"
uses: schneegans/dynamic-badges-action@0e50b8bad39e7e1afd3e4e9c2b7dd145fad07501 # v1.8.0
- name: "Update coverage badges"
if: ${{ github.ref == 'refs/heads/master' && github.repository_owner == 'crowdsecurity' }}
with:
auth: ${{ secrets.GIST_BADGES_SECRET }}
gistID: ${{ secrets.GIST_BADGES_ID }}
filename: crowdsec_parsers_badge.json
label: Hub Parsers
message: ${{ env.PARSERS_COV }}
color: ${{ env.PARSERS_BADGE_COLOR }}
- name: "Create Scenarios badge"
uses: schneegans/dynamic-badges-action@0e50b8bad39e7e1afd3e4e9c2b7dd145fad07501 # v1.8.0
if: ${{ github.ref == 'refs/heads/master' && github.repository_owner == 'crowdsecurity' }}
with:
auth: ${{ secrets.GIST_BADGES_SECRET }}
gistID: ${{ secrets.GIST_BADGES_ID }}
filename: crowdsec_scenarios_badge.json
label: Hub Scenarios
message: ${{ env.SCENARIOS_COV }}
color: ${{ env.SCENARIO_BADGE_COLOR }}
env:
GH_TOKEN: ${{ secrets.GIST_BADGES_SECRET }}
GIST_ID: ${{ secrets.GIST_BADGES_ID }}
run: |
set -euo pipefail
# shields.io endpoint badge, written as a gist file
badge() { jq -nc --arg l "$1" --arg m "$2" --arg c "$3" \
'{schemaVersion:1,label:$l,message:$m,color:$c}'; }
jq -n \
--arg p "$(badge 'Hub Parsers' "$PARSERS_COV" "$PARSERS_BADGE_COLOR")" \
--arg s "$(badge 'Hub Scenarios' "$SCENARIOS_COV" "$SCENARIO_BADGE_COLOR")" \
'{files:{"crowdsec_parsers_badge.json":{content:$p},
"crowdsec_scenarios_badge.json":{content:$s}}}' \
| gh api --method PATCH "/gists/${GIST_ID}" --input -
.github/workflows/bats-mysql.yml
+6
View File
@@ -7,6 +7,9 @@ on:
required: true
type: string
permissions:
contents: read
jobs:
build:
name: "Functional tests"
@@ -32,6 +35,7 @@ jobs:
with:
fetch-depth: 0
submodules: true
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
@@ -48,6 +52,8 @@ jobs:
run: |
make clean bats-build bats-fixture BUILD_STATIC=1
env:
# fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
TEST_HUB_CLONE: "true"
DB_BACKEND: mysql
MYSQL_HOST: 127.0.0.1
MYSQL_PORT: 3306
.github/workflows/bats-postgres.yml
+6
View File
@@ -3,6 +3,9 @@ name: (sub) Bats / Postgres
on:
workflow_call:
permissions:
contents: read
jobs:
build:
name: "Functional tests"
@@ -41,6 +44,7 @@ jobs:
with:
fetch-depth: 0
submodules: true
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
@@ -57,6 +61,8 @@ jobs:
run: |
make clean bats-build bats-fixture BUILD_STATIC=1
env:
# fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
TEST_HUB_CLONE: "true"
DB_BACKEND: pgx
PGHOST: 127.0.0.1
PGPORT: 5432
.github/workflows/bats-sqlite-coverage.yml
+7
View File
@@ -6,6 +6,9 @@ on:
CODECOV_TOKEN:
required: true
permissions:
contents: read
env:
TEST_COVERAGE: true
@@ -27,6 +30,7 @@ jobs:
with:
fetch-depth: 0
submodules: true
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
@@ -40,6 +44,9 @@ jobs:
sudo apt -qq -y -o=Dpkg::Use-Pty=0 install build-essential daemonize jq libre2-dev
- name: "Build crowdsec and fixture"
# fetch the hub index from a git clone to avoid cdn-hub rate limiting (403)
env:
TEST_HUB_CLONE: "true"
run: |
make clean bats-build bats-fixture BUILD_STATIC=1
.github/workflows/bats.yml
+3
View File
@@ -22,6 +22,9 @@ on:
paths-ignore:
- "README.md"
permissions:
contents: read
jobs:
sqlite:
uses: ./.github/workflows/bats-sqlite-coverage.yml
.github/workflows/cache-cleanup.yaml
+6 -20
View File
@@ -6,30 +6,16 @@ on:
types:
- closed
permissions:
actions: write # to delete the branch's Actions caches
jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Cleanup
run: |
gh extension install actions/gh-actions-cache
REPO=${{ github.repository }}
BRANCH="refs/pull/${{ github.event.pull_request.number }}/merge"
echo "Fetching list of cache key"
cacheKeysForPR=$(gh actions-cache list -R $REPO -B $BRANCH | cut -f 1 )
## Setting this to not fail the workflow while deleting cache keys.
set +e
echo "Deleting caches..."
for cacheKey in $cacheKeysForPR
do
gh actions-cache delete $cacheKey -R $REPO -B $BRANCH --confirm
done
echo "Done"
run: gh cache delete --all --ref "refs/pull/${PR_NUMBER}/merge" --succeed-on-no-caches
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_REPO: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
.github/workflows/ci-windows-build-msi.yml
+4
View File
@@ -19,6 +19,9 @@ on:
- build/windows/installer/*.wxs
- .github/workflows/ci-windows-build-msi.yml
permissions:
contents: read
jobs:
build:
name: Build
@@ -31,6 +34,7 @@ jobs:
with:
fetch-depth: 0
submodules: false
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
.github/workflows/codeql-analysis.yml
+1
View File
@@ -48,6 +48,7 @@ jobs:
with:
# required to pick up tags for BUILD_VERSION
fetch-depth: 0
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
.github/workflows/docker-tests.yml
+5
View File
@@ -13,6 +13,10 @@ on:
- releases/**
paths-ignore:
- "README.md"
permissions:
contents: read
env:
CROWDSEC_BYPASS_DB_VOLUME_CHECK: "TRUE" # to avoid the db volume check in the docker_start.sh script
@@ -32,6 +36,7 @@ jobs:
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
.github/workflows/go-tests-windows.yml
+4
View File
@@ -11,6 +11,9 @@ on:
paths-ignore:
- "README.md"
permissions:
contents: read
env:
CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF: true
@@ -25,6 +28,7 @@ jobs:
with:
fetch-depth: 0
submodules: false
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
.github/workflows/go-tests.yml
+13 -5
View File
@@ -16,6 +16,9 @@ on:
paths-ignore:
- "README.md"
permissions:
contents: read
# these env variables are for localstack, so we can emulate aws services
env:
AWS_HOST: localstack
@@ -127,6 +130,7 @@ jobs:
with:
fetch-depth: 0
submodules: true
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
@@ -139,14 +143,18 @@ jobs:
# ensure the version of 'protoc' matches the one that generated the files
PROTOBUF_VERSION="35.0"
JAVY_VERSION="8.1.1"
# sha256 of the release assets, verified before use
PROTOBUF_SHA256="a45cda0989c17dd950db55f6fbe1e5814c50fda08e87aa422980ac1f89dddbbc"
JAVY_SHA256="4bec224e1d51827808fd063541a2b4d94cf474744636b692be63e7a29bec83a6"
# don't pollute the repo
pushd $HOME
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip
unzip protoc-${PROTOBUF_VERSION}-linux-x86_64.zip -d $HOME/.protoc
curl -OL https://github.com/bytecodealliance/javy/releases/download/v${JAVY_VERSION}/javy-x86_64-linux-v${JAVY_VERSION}.gz
curl --fail --location --proto '=https' --tlsv1.2 -o protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip
echo "${PROTOBUF_SHA256} protoc.zip" | sha256sum -c -
unzip protoc.zip -d $HOME/.protoc
curl --fail --location --proto '=https' --tlsv1.2 -o javy.gz https://github.com/bytecodealliance/javy/releases/download/v${JAVY_VERSION}/javy-x86_64-linux-v${JAVY_VERSION}.gz
echo "${JAVY_SHA256} javy.gz" | sha256sum -c -
mkdir -p $HOME/.javy
gzip -d javy-x86_64-linux-v${JAVY_VERSION}.gz
mv javy-x86_64-linux-v${JAVY_VERSION} $HOME/.javy/javy
gzip -dc javy.gz > $HOME/.javy/javy
chmod +x $HOME/.javy/javy
popd
export PATH="$HOME/.protoc/bin:$HOME/.javy/:$PATH"
.github/workflows/publish-docker-master.yml
+4 -6
View File
@@ -17,6 +17,10 @@ on:
- 'go.sum'
- 'Makefile'
permissions:
contents: read
packages: write # push images to ghcr.io
jobs:
dev-alpine:
uses: ./.github/workflows/publish-docker.yml
@@ -28,9 +32,6 @@ jobs:
push: true
slim: false
debian: false
secrets:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
dev-debian:
uses: ./.github/workflows/publish-docker.yml
@@ -42,6 +43,3 @@ jobs:
push: true
slim: false
debian: true
secrets:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
.github/workflows/publish-docker-release.yml
+4 -6
View File
@@ -18,12 +18,13 @@ on:
default: false
required: true
permissions:
contents: read
packages: write # push images to ghcr.io
jobs:
alpine:
uses: ./.github/workflows/publish-docker.yml
secrets:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
with:
image_version: ${{ github.event.inputs.image_version }}
crowdsec_version: ${{ github.event.inputs.crowdsec_version }}
@@ -35,9 +36,6 @@ jobs:
debian:
uses: ./.github/workflows/publish-docker.yml
secrets:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
with:
image_version: ${{ github.event.inputs.image_version }}
crowdsec_version: ${{ github.event.inputs.crowdsec_version }}
.github/workflows/publish-docker.yml
+10 -5
View File
@@ -2,12 +2,11 @@ name: (sub) Publish Docker images
on:
workflow_call:
secrets:
DOCKER_USERNAME:
required: true
DOCKER_PASSWORD:
required: true
inputs:
environment:
required: false
type: string
default: docker
platform:
required: true
type: string
@@ -30,15 +29,21 @@ on:
required: true
type: boolean
permissions:
contents: read
packages: write # push images to ghcr.io
jobs:
push_to_registry:
name: Push Docker image to registries
runs-on: crowdsec-large-runner
environment: ${{ inputs.environment }}
steps:
- name: Check out the repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Set up QEMU
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
.github/workflows/publish-tarball-release.yml
+1
View File
@@ -20,6 +20,7 @@ jobs:
with:
fetch-depth: 0
submodules: false
persist-credentials: false
- name: "Set up Go"
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
.github/workflows/update_docker_hub_doc.yml
+5
View File
@@ -7,6 +7,9 @@ on:
paths:
- 'build/docker/README.md'
permissions:
contents: read
jobs:
update-docker-hub-readme:
runs-on: ubuntu-latest
@@ -16,6 +19,8 @@ jobs:
name: Check out the repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
if: ${{ github.repository_owner == 'crowdsecurity' }}
with:
persist-credentials: false
-
name: Update docker hub README
.github/workflows/version.yml
+9 -1
View File
@@ -14,6 +14,9 @@ on:
required: false
type: string
permissions:
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
@@ -22,6 +25,8 @@ jobs:
(github.event_name == 'release' && github.event.release.prerelease == false)
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
@@ -55,10 +60,13 @@ jobs:
echo "date=$DATE" >> "$GITHUB_OUTPUT"
- name: Write dist artifact
env:
TAG: ${{ steps.pick_version.outputs.tag }}
DATE: ${{ steps.pick_version.outputs.date }}
run: |
set -euo pipefail
mkdir -p ./dist
echo '{"name":"${{ steps.pick_version.outputs.tag }}", "tag_name":"${{ steps.pick_version.outputs.tag }}", "published_at":"${{ steps.pick_version.outputs.date }}"}' > ./dist/latest
printf '{"name":"%s", "tag_name":"%s", "published_at":"%s"}\n' "$TAG" "$TAG" "$DATE" > ./dist/latest
cat ./dist/latest
# Deploy the build output directory (adjust ./dist to your framework output)
test/lib/config/config-local
+30 -1
View File
@@ -112,6 +112,35 @@ config_generate() {
yq -i e '.cscli.hub_branch=strenv(HUB_BRANCH)' "${CONFIG_DIR}/config.yaml"
}
# Populate the hub index. By default this pulls it from the hub CDN, but the
# CDN rate-limits shared CI IPs (403). Set TEST_HUB_CLONE=1 to instead fetch the
# index from a git clone of the hub at the pinned branch: the committed
# .index.json already embeds every item's content, so 'collections install'
# needs no further downloads.
# Print the runner's public IP, to debug hub CDN / git rate limiting (403).
log_public_ip() {
local ip
ip=$(curl -fsS --max-time 10 https://api.ipify.org 2>/dev/null) || ip="unknown"
echo >&2 "hub index fetch failed - runner public IP: ${ip}"
}
hub_update() {
if [[ -z "${TEST_HUB_CLONE:-}" ]]; then
"$CSCLI" --warning hub update --with-content || { log_public_ip; return 1; }
return
fi
echo "Fetching hub index from a git clone (branch $HUB_BRANCH)"
local clonedir
clonedir=$(mktemp -d)
# shellcheck disable=SC2064
trap "rm -rf '$clonedir'" RETURN
git clone --quiet --depth 1 --branch "$HUB_BRANCH" \
"${TEST_HUB_GIT_URL:-https://github.com/crowdsecurity/hub.git}" "$clonedir" \
|| { log_public_ip; return 1; }
cp "$clonedir/.index.json" "$HUB_DIR/.index.json"
}
make_init_data() {
./bin/assert-crowdsec-not-running || die "Cannot create fixture data."
@@ -126,7 +155,7 @@ make_init_data() {
./instance-db config-yaml
./instance-db setup
"$CSCLI" --warning hub update --with-content
hub_update
# preload some content and data files
"$CSCLI" collections install crowdsecurity/linux --download-only