third_party_libbpf/.travis.yml

131 lines
5.4 KiB
YAML
Raw Normal View History

sudo: required
libbpf: Add VMTEST to CI Extend continuous integration tests by adding testing against various kernel versions. The code is based on vmtest CI scripts implemented by osandov@ for drgn [1] with the following modifications: - The downloadables are stored in Amazon S3 cloud indexed in [2] - `--setup-cmd` command line option is added to vmtest/run.sh so setup commands run on VM boot can be set in e.g. `.travis.yml` - Travis build matrix [2] is introduced for VM tests so VM tests are followed by the existing CI tests. The matrix has `KERNEL` and `VMTEST_SETUPCMD` dimensions. - Minor style fixes. The vmtest extention code is located in travis-ci/vmtest and contains `run.sh` and `setup_example.sh` - `run.sh` is responsible for the vmtest workflow: downloading vmlinux and rootfs image from the cloud, fs mounting, syncing libbpf sources to the image, setting up scripts run on VM boot, starting VM using QEMU. `run.sh` covers more use cases than a script for a job run in TravisCI, e.g. int can build a kernel w/ `--build` option. - `setup_example.sh` is an example of a script run in VM which can be modified to e.g. run actual libbpf tests. A setup script should have executable permission. To set up a new kernel version for a test: 1) upload vmlinuz.* and vmlinux.*\.zst to Amazon S3 store located at [4]; 2) modify INDEX [2] file. [1] https://github.com/osandov/drgn [2] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/INDEX [3] https://docs.travis-ci.com/user/build-matrix [4] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/
2019-11-15 19:04:51 +00:00
language: bash
dist: focal
services:
- docker
env:
global:
libbpf: Add VMTEST to CI Extend continuous integration tests by adding testing against various kernel versions. The code is based on vmtest CI scripts implemented by osandov@ for drgn [1] with the following modifications: - The downloadables are stored in Amazon S3 cloud indexed in [2] - `--setup-cmd` command line option is added to vmtest/run.sh so setup commands run on VM boot can be set in e.g. `.travis.yml` - Travis build matrix [2] is introduced for VM tests so VM tests are followed by the existing CI tests. The matrix has `KERNEL` and `VMTEST_SETUPCMD` dimensions. - Minor style fixes. The vmtest extention code is located in travis-ci/vmtest and contains `run.sh` and `setup_example.sh` - `run.sh` is responsible for the vmtest workflow: downloading vmlinux and rootfs image from the cloud, fs mounting, syncing libbpf sources to the image, setting up scripts run on VM boot, starting VM using QEMU. `run.sh` covers more use cases than a script for a job run in TravisCI, e.g. int can build a kernel w/ `--build` option. - `setup_example.sh` is an example of a script run in VM which can be modified to e.g. run actual libbpf tests. A setup script should have executable permission. To set up a new kernel version for a test: 1) upload vmlinuz.* and vmlinux.*\.zst to Amazon S3 store located at [4]; 2) modify INDEX [2] file. [1] https://github.com/osandov/drgn [2] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/INDEX [3] https://docs.travis-ci.com/user/build-matrix [4] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/
2019-11-15 19:04:51 +00:00
- PROJECT_NAME='libbpf'
2019-10-31 12:15:15 +00:00
- AUTHOR_EMAIL="$(git log -1 --pretty=\"%aE\")"
- REPO_ROOT="$TRAVIS_BUILD_DIR"
- CI_ROOT="$REPO_ROOT/travis-ci"
- VMTEST_ROOT="$CI_ROOT/vmtest"
libbpf: Add VMTEST to CI Extend continuous integration tests by adding testing against various kernel versions. The code is based on vmtest CI scripts implemented by osandov@ for drgn [1] with the following modifications: - The downloadables are stored in Amazon S3 cloud indexed in [2] - `--setup-cmd` command line option is added to vmtest/run.sh so setup commands run on VM boot can be set in e.g. `.travis.yml` - Travis build matrix [2] is introduced for VM tests so VM tests are followed by the existing CI tests. The matrix has `KERNEL` and `VMTEST_SETUPCMD` dimensions. - Minor style fixes. The vmtest extention code is located in travis-ci/vmtest and contains `run.sh` and `setup_example.sh` - `run.sh` is responsible for the vmtest workflow: downloading vmlinux and rootfs image from the cloud, fs mounting, syncing libbpf sources to the image, setting up scripts run on VM boot, starting VM using QEMU. `run.sh` covers more use cases than a script for a job run in TravisCI, e.g. int can build a kernel w/ `--build` option. - `setup_example.sh` is an example of a script run in VM which can be modified to e.g. run actual libbpf tests. A setup script should have executable permission. To set up a new kernel version for a test: 1) upload vmlinuz.* and vmlinux.*\.zst to Amazon S3 store located at [4]; 2) modify INDEX [2] file. [1] https://github.com/osandov/drgn [2] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/x86_64/INDEX [3] https://docs.travis-ci.com/user/build-matrix [4] https://libbpf-vmtest.s3-us-west-1.amazonaws.com/
2019-11-15 19:04:51 +00:00
addons:
apt:
packages:
- qemu-kvm
- zstd
vmtest: build and run bpf kernel selftests against various kernels Run kernel selftests in vmtest with the goal to test libbpf backward compatibility with older kernels. The list of kernels should be specified in .travis.yml config in `jobs` section, e.g. KERNEL=5.5.0. Enlisted kernel releases - 5.5.0 # built from main - 5.5.0-rc6 # built from bpf-next - LATEST The kernel specified as 'LATEST' in .travis.yml is built from bpf-next kernel tree, the rest of the kernels are downloaded from the specified in INDEX file. The kernel sources from bpf-next are manually patched with [1] from bpf tree to fix ranqslower build. This workaround should be removed after the patch is merged from bpf to bpf-next tree. Due to kernel sources being checked out the duration of the LATEST kernel test is ~30m. bpf selftests are built from tools/testing/selftests/bpf/ of bpf-next tree with HEAD revision set to CHECKPOINT-COMMIT specified in libbpf so selftests and libbpf are in sync. Currently only programs are tested with test_progs program, test_maps and test_verifier should follow. test_progs are run with blacklist required due to: - some features, e.g. fentry/fexit are not supported in older kernels - environment limitations, e.g an absence of the recent pahole in Debian - incomplete disk image The blacklist is passed to test_progs with -b option as specified in [2] patch set. Most of the preceeding tests are disabled due to incomplete disk image currenly lacking proper networking settings. For the LATEST kernel fome fentry/fexit tests are disabled due to pahole v1.16 is not abailible in Debian yet. Next steps are resolving issues with blacklisted tests, enabling maps and verifier testing, expanding the list of tested kernels. [1] https://lore.kernel.org/bpf/908498f794661c44dca54da9e09dc0c382df6fcb.1580425879.git.hex@fb.com/t.mbox.gz [2] https://www.spinics.net/lists/netdev/msg625192.html
2020-02-12 20:20:53 +00:00
- binutils-dev
- elfutils
- libcap-dev
- libelf-dev
- libdw-dev
2019-10-31 12:15:15 +00:00
stages:
# Run Coverity periodically instead of for each PR for following reasons:
# 1) Coverity jobs are heavily rate-limited
# 2) Due to security restrictions of encrypted environment variables
# in Travis CI, pull requests made from forks can't access encrypted
# env variables, making Coverity unusable
# See: https://docs.travis-ci.com/user/pull-requests#pull-requests-and-security-restrictions
- name: Coverity
if: type = cron
jobs:
include:
- stage: Builds & Tests
name: Kernel 5.5.0 + selftests
language: bash
env: KERNEL=5.5.0
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Kernel LATEST + selftests
language: bash
env: KERNEL=LATEST
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Kernel 4.9.0 + selftests
language: bash
env: KERNEL=4.9.0
script: $CI_ROOT/vmtest/run_vmtest.sh || travis_terminate 1
- name: Debian Build
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Debian Build (ASan+UBSan)
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN_ASAN || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Debian Build (clang)
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN_CLANG || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Debian Build (clang ASan+UBSan)
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN_CLANG_ASAN || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Debian Build (gcc-10)
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN_GCC10 || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Debian Build (gcc-10 ASan+UBSan)
language: bash
install: $CI_ROOT/managers/debian.sh SETUP
script: $CI_ROOT/managers/debian.sh RUN_GCC10_ASAN || travis_terminate 1
after_script: $CI_ROOT/managers/debian.sh CLEANUP
- name: Ubuntu Focal Build
language: bash
script: sudo $CI_ROOT/managers/ubuntu.sh || travis_terminate 1
2019-10-23 08:42:21 +00:00
- name: Ubuntu Focal Build (arm)
2019-10-23 08:42:21 +00:00
arch: arm64
language: bash
script: sudo $CI_ROOT/managers/ubuntu.sh || travis_terminate 1
2019-10-31 12:15:15 +00:00
- name: Ubuntu Focal Build (s390x)
arch: s390x
language: bash
script: sudo $CI_ROOT/managers/ubuntu.sh || travis_terminate 1
- name: Ubuntu Focal Build (ppc64le)
2019-12-10 10:35:24 +00:00
arch: ppc64le
language: bash
script: sudo $CI_ROOT/managers/ubuntu.sh || travis_terminate 1
2019-10-31 12:15:15 +00:00
- stage: Coverity
language: bash
env:
# Coverity configuration
# COVERITY_SCAN_TOKEN=xxx
# Encrypted using `travis encrypt --repo libbpf/libbpf COVERITY_SCAN_TOKEN=xxx`
- secure: "I9OsMRHbb82IUivDp+I+w/jEQFOJgBDAqYqf1ollqCM1QhocxMcS9bwIAgfPhdXi2hohV7sRrVMZstahY67FAvJLGxNopi4tAPDIAaIFxgO0yDxMhaTMx5xDfMwlIm2FOP/9gB9BQsd6M7CmoQZgXYwBIv7xd1ooxoQrh2rOK1YrRl7UQu3+c3zPTjDfIYZzR3bFttMqZ9/c4U0v8Ry5IFXrel3hCshndHA1TtttJrUSrILlZcmVc1ch7JIy6zCbCU/2lGv0B/7rWXfF8MT7O9jPtFOhJ1DEcd2zhw2n4j9YT3a8OhtnM61LA6ask632mwCOsxpFLTun7AzuR1Cb5mdPHsxhxnCHcXXARa2mJjem0QG1NhwxwJE8sbRDapojexxCvweYlEN40ofwMDSnj/qNt95XIcrk0tiIhGFx0gVNWvAdmZwx+N4mwGPMTAN0AEOFjpgI+ZdB89m+tL/CbEgE1flc8QxUxJhcp5OhH6yR0z9qYOp0nXIbHsIaCiRvt/7LqFRQfheifztWVz4mdQlCdKS9gcOQ09oKicPevKO1L0Ue3cb7Ug7jOpMs+cdh3XokJtUeYEr1NijMHT9+CTAhhO5RToWXIZRon719z3fwoUBNDREATwVFMlVxqSO/pbYgaKminigYbl785S89YYaZ6E5UvaKRHM6KHKMDszs="
- COVERITY_SCAN_PROJECT_NAME="libbpf"
- COVERITY_SCAN_NOTIFICATION_EMAIL="${AUTHOR_EMAIL}"
- COVERITY_SCAN_BRANCH_PATTERN="$TRAVIS_BRANCH"
# Note: `make -C src/` as a BUILD_COMMAND will not work here
- COVERITY_SCAN_BUILD_COMMAND_PREPEND="cd src/"
- COVERITY_SCAN_BUILD_COMMAND="make"
install:
- sudo echo 'deb-src http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse' >>/etc/apt/sources.list
- sudo apt-get update
- sudo apt-get -y build-dep libelf-dev
- sudo apt-get install -y libelf-dev pkg-config
2019-10-31 12:15:15 +00:00
script:
- scripts/coverity.sh || travis_terminate 1
allow_failures:
- env: KERNEL=x.x.x