Commit Graph

97 Commits

Author SHA1 Message Date
Peter Maydell
757acb9a82 * Move unit and bench tests into separate directories
* Clean-up and improve gitlab-ci jobs
 * Drop the non-working "check-speed" makefile target
 * Minor documentation updates
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmBLonURHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbVsZg/9HoRP+nUoz9y7d2/GpXP01zj5xtatMadW
 RFUAlCqJapWyDXlzXVlKmvjIQeBa1uvLv6eBbAr9E748Aurmwrpw+TZ0rvIgbZrG
 fNYwixBjWcf8BtxoqRki/zdjsBV9Lym7H0G/QxYxOfve8IdHntE5iVnI0hCpaNge
 E2SLAXlFsKvC7MbLtm7+KRyIyA4PAshqJWgH+T2mdOcAA6SudLU4SidyPQJo02LX
 8U2NP2zB/5VldFGtc3TuFmoWhCSctfa3ntmNTBUBfJAEoB7hUTdKH3naZ52Qd2X9
 nAEWJAh7yiXnClz8Ant/QywvM+OjxmxVWz2Kc/1jJYqLQVGB1e8JBWH6pJxboDJb
 8z+G5SYAw5woOE+ya57doPyDbi9EHn5O6K5FIjhn/rc5YJp6SA9ilG89E3vrhr/m
 RMTCvm2nrPxenXXSovKi1t5SB6X8uyNKOuAXgEef8T28c03ZSZbs6u/YTDjPHSkp
 lU9PGxr+Ft2on+GzSXoC1HBzLyesiTsjF7leE4aB9jOGgnqCMiJsALyx/XWq5x1m
 wtpp7TcObB7p9DZ2Kf4601CoVaVO2ESjor9T81JLDtqVrlD0VHhuFGsFvOlo/M2k
 JePmg93CoX+ptP47Pyou8fnjkwCI4raM55s1SsYvVqdg2Rs1Btj6CwIqc2QEjn7N
 KIXZ7HG4i80=
 =YxBC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-03-12' into staging

* Move unit and bench tests into separate directories
* Clean-up and improve gitlab-ci jobs
* Drop the non-working "check-speed" makefile target
* Minor documentation updates

# gpg: Signature made Fri 12 Mar 2021 17:18:45 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-03-12:
  README: Add Documentation blurb
  MAINTAINERS: Merge the Gitlab-CI section into the generic CI section
  tests: remove "make check-speed" in favor of "make bench"
  gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs
  gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job
  gitlab-ci.yml: Add some missing dependencies to the jobs
  gitlab-ci.yml: Move build-tools-and-docs-debian to a better place
  tests: Move benchmarks into a separate folder
  tests: Move unit tests into a separate directory

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-14 15:13:53 +00:00
Thomas Huth
342409564c gitlab-ci.yml: Merge check-crypto-old jobs into the build-crypto-old jobs
Both, the build-crypto-old and the check-crypto-old jobs finish reasonably
fast, and the build artifacts are only used for the single corresponding
check jobs, so there is no reason for doing the check step in a separate
job here. Thus let's stop wasting artifacts space and job scheduler over-
head by simply merging the test step into the build jobs.

Message-Id: <20210311142211.1547864-5-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth
7da153e818 gitlab-ci.yml: Merge one of the coroutine jobs with the tcg-disabled job
Our gitlab-ci got quite slow in the past weeks, due to the immense amount
of jobs that we have, so we should try to reduce the number of jobs.
Since we already have a job that builds without TCG, we can merge
one of the "build-coroutine" jobs with it to get rid of at least one
job.

Message-Id: <20210311142211.1547864-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth
4a859abd1d gitlab-ci.yml: Add some missing dependencies to the jobs
Let's make sure that all jobs have proper "needs:" statements so that
they can start as soon as possible, without having to wait for the
previous pipeline stage to finish.

Message-Id: <20210311142211.1547864-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Thomas Huth
dd188e4184 gitlab-ci.yml: Move build-tools-and-docs-debian to a better place
The "build-tools-and-docs-debian" job had been added in between
the "check-system-debian" and the "accepance-system-debian" jobs
and thus separates the jobs that belong together. Move it away,
to the end of the file, next to the "pages" job that depends on it.
And while we're at it, also add a proper "needs:" line to the
job so that it can be started as soon as possible instead of always
waiting for the previous stage to finish.

Message-Id: <20210311142211.1547864-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-12 15:46:30 +01:00
Peter Maydell
f4abdf3271 Testing, guest-loader and other misc tweaks
- add warning text to quickstart example
   - add CFI tests to CI
   - use --arch-only for docker pre-requisites
   - fix .editorconfig for emacs
   - add guest-loader for Xen-like hypervisor testing
   - move generic-loader docs into manual proper
   - move semihosting out of hw/
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmBI50MACgkQ+9DbCVqe
 KkSyKggAhPZW+7sReVEsFdnVfwuo3evW7auoW44mghNbikTnm3RfoahYTrek8lGZ
 AEo2gFMbzENW0j88e0OvSYYtwkVz3sD68bygfXerti6sQwWlwkf42I/suWjJNLph
 oVKGEEdJess9+zR13Cu6RAq5RaTwzDPGPjUwTbeJPpAps4+UZV3hsxhaxs8keII6
 GBa/idnh0qEApP2NDLKiSASrYZM7xGvljE7zO4qhchd6iSH/o5rCtkoB2tRCcXGo
 +KF8LyBsUNf7GiWp0yYZMZUQ3Pqskqma8N3d2A4UlS1kXvxeX/FiORkG/Ne8bH1Z
 VZ1Z/xbyXGlVkiP1bcoYSc6XWHNDTw==
 =R9zQ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-docs-xen-updates-100321-2' into staging

Testing, guest-loader and other misc tweaks

  - add warning text to quickstart example
  - add CFI tests to CI
  - use --arch-only for docker pre-requisites
  - fix .editorconfig for emacs
  - add guest-loader for Xen-like hypervisor testing
  - move generic-loader docs into manual proper
  - move semihosting out of hw/

# gpg: Signature made Wed 10 Mar 2021 15:35:31 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-docs-xen-updates-100321-2:
  semihosting: Move hw/semihosting/ -> semihosting/
  semihosting: Move include/hw/semihosting/ -> include/semihosting/
  tests/avocado: add boot_xen tests
  docs: add some documentation for the guest-loader
  docs: move generic-loader documentation into the main manual
  hw/core: implement a guest-loader to support static hypervisor guests
  device_tree: add qemu_fdt_setprop_string_array helper
  hw/riscv: migrate fdt field to generic MachineState
  hw/board: promote fdt from ARM VirtMachineState to MachineState
  .editorconfig: update the automatic mode setting for Emacs
  tests/docker: Use --arch-only when building Debian cross image
  gitlab-ci.yml: Add jobs to test CFI flags
  gitlab-ci.yml: Allow custom # of parallel linkers
  tests/docker: add a test-tcg for building then running check-tcg
  docs/system: add a gentle prompt for the complexity to come

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-11 16:20:58 +00:00
Daniele Buono
1bb12e172a gitlab-ci.yml: Add jobs to test CFI flags
QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produced is working fine.

The three sets allow testing of x86_64 binaries for x86_64, s390x,
ppc64 and aarch64 targets

[AJB: tweak job names to avoid brands]

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>
2021-03-10 15:34:11 +00:00
Daniele Buono
e6d27a9c0d gitlab-ci.yml: Allow custom # of parallel linkers
Define a new variable LD_JOBS, that can be used to select
the maximum number of linking jobs to be executed in parallel.
If the variable is not defined, maintain the default given by
make -j

Currently, make parallelism at build time is based on the number
of cpus available.

This doesn't work well with LTO at linking, because with LTO the
linker has to load in memory all the intermediate object files
for optimization.
The end result is that, if the gitlab runner happens to run two
linking processes at the same time, the job will fail with an
out-of-memory error,

This patch leverages the ability to maintain high parallelism at
compile time, but limit the number of linkers executed in parallel.

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210304030948.9367-2-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-8-alex.bennee@linaro.org>
2021-03-10 15:34:11 +00:00
Thomas Huth
2cc1a90166 Remove deprecated target tilegx
TILE-Gx was only implemented in linux-user mode, but support for this CPU
was removed from the upstream Linux kernel in 2018, and it has also been
dropped from glibc, so there is no new Linux development taking place with
this architecture. For running the old binaries, users can simply use older
versions of QEMU.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210224183952.80463-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 11:26:32 +01:00
Philippe Mathieu-Daudé
9f3a24cf1f docker: Alpine build job depends on Alpine container
Add missing dependency build-system-alpine -> amd64-alpine-container.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210303130646.1494015-2-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Thomas Huth
53c915c627 gitlab-ci.yml: Run check-tcg with TCI
It's now possible to also run the non-x86 TCG tests with TCI.

Message-Id: <20210127055903.40148-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Philippe Mathieu-Daudé
3473f6ed88 gitlab-ci: Disable vhost-kernel in build-disable job
Commit 299e6f19b3 ("vhost-net: revamp configure logic") added
the --enable-vhost-kernel option.
Disable it in the build-disable job.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210131104621.221602-1-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Philippe Mathieu-Daudé
c071f19b0d gitlab-ci: Display Avocado log content when tests timeout
Since commit ba2232bae6 ("gitlab-ci: Refactor code that show logs
of failed acceptances") we display the log content of failing tests
(Avocado "FAIL" event).

Since we are also interested in tests timeouting, update our global
Avocado config to display log content for the "INTERRUPT" event,
"possible when the timeout is reached" (See [*]).

[*] https://avocado-framework.readthedocs.io/en/latest/guides/writer/chapters/writing.html#test-statuses

Suggested-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210215171438.935665-1-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Daniel P. Berrangé
f5157dc770 gitlab: fix inconsistent indentation
The standard is to use 2 space indent, not 3.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Daniel P. Berrangé
764a0747a7 gitlab: add fine grained job deps for all build jobs
This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Thomas Huth
0c99f50c52 travis.yml: Move the -fsanitize=thread testing to the gitlab-CI
Use clang-10, so we can also use the --enable-tsan configure
option instead of only passing the flag via --extra-cflags.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-6-thuth@redhat.com>
Message-Id: <20210211122750.22645-6-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth
ac6d7074c0 travis.yml: (Re-)move the --enable-debug jobs
We already have similar jobs in the gitlab-CI ("build-some-softmmu" and
"build-user-plugins"), so let's switch one of them to use --enable-debug
instead of --enable-debug-tcg, then we can simply drop these jobs from
the Travis-CI.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210211045455.456371-5-thuth@redhat.com>
Message-Id: <20210211122750.22645-5-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth
3382261c26 travis.yml: Move the --enable-modules test to the gitlab-CI
Simply add the flag to an existing job, no need for yet another
job here.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-4-thuth@redhat.com>
Message-Id: <20210211122750.22645-4-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Thomas Huth
c2b33c43fd travis.yml: Move the -fsanitize=undefined test to the gitlab-CI
Add it to the existing Clang job and also add a job that covers the
linux-user code with this compiler flag. To make sure that the detected
problems are not simply ignored, let's also use "-fno-sanitize-recover=..."
now instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210211045455.456371-3-thuth@redhat.com>
Message-Id: <20210211122750.22645-3-alex.bennee@linaro.org>
2021-02-15 09:38:26 +00:00
Philippe Mathieu-Daudé
04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Stefan Hajnoczi
b6053de79f gitlab-ci: remove redundant GitLab repo URL command
It is no longer necessary to point .gitmodules at GitLab repos when
running in GitLab CI since they are now used all the time.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-4-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-09 20:53:56 +00:00
Thomas Huth
12a917a14f gitlab-ci.yml: Avoid recompiling the sources in the test jobs
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.

In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:40 +01:00
Thomas Huth
8cd95aec4c gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.

Message-Id: <20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:13 +01:00
Thomas Huth
db0108d5d8 gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.

While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.

Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Philippe Mathieu-Daudé
4e41d4a34e gitlab-ci: Test building linux-user targets on CentOS 7
Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:

  [705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  ../linux-user/strace.c: In function 'do_print_sockopt':
  ../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
           case IPV6_ADDR_PREFERENCES:
                ^

This job currently takes 31 minutes 32 seconds ([4]).

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/978223286

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 15:39:34 +01:00
Jiaxun Yang
b22786eab0 gitlab-ci: Add alpine to pipeline
[thuth: Enable "make check" tests, too, after tests/check-block.sh got fixed]

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.12471-10-jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-20 10:46:54 +01:00
Peter Maydell
b93f4fbdc4 docs: Build and install all the docs in a single manual
When we first converted our documentation to Sphinx, we split it into
multiple manuals (system, interop, tools, etc), which are all built
separately.  The primary driver for this was wanting to be able to
avoid shipping the 'devel' manual to end-users.  However, this is
working against the grain of the way Sphinx wants to be used and
causes some annoyances:
 * Cross-references between documents become much harder or
   possibly impossible
 * There is no single index to the whole documentation
 * Within one manual there's no links or table-of-contents info
   that lets you easily navigate to the others
 * The devel manual doesn't get published on the QEMU website
   (it would be nice to able to refer to it there)

Merely hiding our developer documentation from end users seems like
it's not enough benefit for these costs.  Combine all the
documentation into a single manual (the same way that the readthedocs
site builds it) and install the whole thing.  The previous manual
divisions remain as the new top level sections in the manual.

 * The per-manual conf.py files are no longer needed
 * The man_pages[] specifications previously in each per-manual
   conf.py move to the top level conf.py
 * docs/meson.build logic is simplified as we now only need to run
   Sphinx once for the HTML and then once for the manpages5B
 * The old index.html.in that produced the top-level page with
   links to each manual is no longer needed

Unfortunately this means that we now have to build the HTML
documentation into docs/manual in the build tree rather than directly
into docs/; otherwise it is too awkward to ensure we install only the
built manual and not also the dependency info, stamp file, etc.  The
manual still ends up in the same place in the final installed
directory, but anybody who was consulting documentation from within
the build tree will have to adjust where they're looking.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20210115154449.4801-1-peter.maydell@linaro.org
2021-01-19 15:45:14 +00:00
Alex Bennée
22250a6ecd gitlab: migrate the minimal tools and unit tests from Travis
These tests are good at shaking out missing stubs which otherwise work
if we have built targets. Rather than create a new job just add the
checks to the existing tools-and-docs build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210114165730.31607-12-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Alex Bennée
a8a3abe0b3 gitlab: move docs and tools build across from Travis
While we are at it we might as well check the tag generation. For
bonus points we run GNU globals htags into the public pages directory
for publishing with the auto generated pages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210114165730.31607-10-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Cho, Yu-Chen
b115ea3a0d gitlab-ci.yml: Add openSUSE Leap 15.2 for gitlab CI/CD
Add build-system-opensuse jobs and opensuse-leap.docker dockerfile.
Use openSUSE Leap 15.2 container image in the gitlab-CI.

Signed-off-by: Cho, Yu-Chen <acho@suse.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201229085046.8536-1-acho@suse.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:08 +01:00
Alex Bennée
53f41245b0 gitlab: add --without-default-features build
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201210190417.31673-8-alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Alex Bennée
3fed93f312 gitlab: move --without-default-devices build from Travis
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201210190417.31673-7-alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Wainer dos Santos Moschetta
df063546c2 gitlab-ci: Archive logs of acceptance tests
Keep the logs of acceptance tests for two days on GitLab. If you want
to make it available for more time, click on the 'Keep' button on
the Job page at web UI.

By default GitLab will archive artifacts only if the job succeed.
Instead let's keep it on both success and failure, so it gives the
opportunity to the developer/maintainer to check the error logs
as well as to the logs of CANCEL tests (not shown on the job logs).

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201211183827.915232-4-wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-18 12:25:42 +01:00
Wainer dos Santos Moschetta
ba2232bae6 gitlab-ci: Refactor code that show logs of failed acceptances
Replace the code (python) on after_script of the acceptance jobs that
is currently used to show the logs of failed tests. Instead it is used
the Avocado's testlogs plug-in which works likewise.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20201211183827.915232-3-wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-18 12:25:42 +01:00
Peter Maydell
5e7b204dbf pc,pci,virtio: fixes, cleanups
Lots of fixes, cleanups.
 CPU hot-unplug improvements.
 A new AER property for virtio devices, adding a dummy AER capability.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAl/REawPHG1zdEByZWRo
 YXQuY29tAAoJECgfDbjSjVRp2MQIALsT4+JQRvo3YR0L3hpVFzRU4cMwVYryqe2/
 RRWggBI4OB3kiQT2YD4lElXmZnTTG5p5B69P+DE5Wj5faTQoIQdWZjDS9pO+tI+p
 gVNKgaN7ByKz5PUQNhhGKUfQ4F6x5yDePWlWGJZAmExEysAuC/nmCe5Lo1PrDzEV
 t0H/TmCNxAb4HsT9LG1pogTM/lpXDsjlVG/Mryh2XNe7z6c0ZyGo1WMI59RFVGId
 9EpOV1OXWVHgkSWD6Rn7mqNq7zASS2Du/osHcQ6C51L/UE+pZPJW7gDqDRudItBR
 XXteNAeLXo5NdGJsTdVuHfy+Z0fJdHny9jLHMO2FkhV1wzYgJSM=
 =LaYX
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

pc,pci,virtio: fixes, cleanups

Lots of fixes, cleanups.
CPU hot-unplug improvements.
A new AER property for virtio devices, adding a dummy AER capability.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Wed 09 Dec 2020 18:04:28 GMT
# gpg:                using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg:                issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream: (65 commits)
  hw/virtio-pci Added AER capability.
  hw/virtio-pci Added counter for pcie capabilities offsets.
  pcie_aer: Fix help message of pcie_aer_inject_error command
  x86: ich9: let firmware negotiate 'CPU hot-unplug with SMI' feature
  x86: ich9: factor out "guest_cpu_hotplug_features"
  tests/acpi: update expected files
  x86: acpi: let the firmware handle pending "CPU remove" events in SMM
  tests/acpi: allow expected files change
  x86: acpi: introduce AcpiPmInfo::smi_on_cpu_unplug
  acpi: cpuhp: introduce 'firmware performs eject' status/control bits
  hw/i386/pc: add max combined fw size as machine configuration option
  block/export: avoid g_return_val_if() input validation
  contrib/vhost-user-input: avoid g_return_val_if() input validation
  contrib/vhost-user-gpu: avoid g_return_val_if() input validation
  contrib/vhost-user-blk: avoid g_return_val_if() input validation
  .gitlab-ci: add build-libvhost-user
  libvhost-user: add a simple link test without glib
  libvhost-user: make it a meson subproject
  libvhost-user: drop qemu/osdep.h dependency
  libvhost-user: remove qemu/compiler.h usage
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-12-09 20:08:54 +00:00
Marc-André Lureau
c4698e360c .gitlab-ci: add build-libvhost-user
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20201125100640.366523-9-marcandre.lureau@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-12-09 13:04:17 -05:00
Philippe Mathieu-Daudé
d1a8b308a4 gitlab-ci: Move coroutine tests across to gitlab
Similarly to commit 8cdb2cef3f, move the coroutine tests to GitLab.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201108204535.2319870-13-philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
[thuth: Replaced Travis by Gitlab-CI in comment]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé
d01bc2531c gitlab-ci: Move user-static test across to gitlab
Similarly to commit 8cdb2cef3f, move the user-static test to GitLab.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201108204535.2319870-11-philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé
0c4d885f08 gitlab-ci: Update 'build-disabled' to cover all configurable options
Add these missing options to the 'build-disabled' job:
  --disable-auth-pam             (commit 8953caf3cd)
  --disable-gcrypt               (commit 91bfcdb01d)
  --disable-keyring              (commit 54e7aac056)
  --disable-libudev              (commit 5c53015a48)
  --disable-opengl               (commit da076ffed6)
  --disable-sparse               (commit 03b4fe7de2)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[thuth: Dropped the unnecessary hunk with --disable-vhost-user-blk-server]
Message-Id: <20201108204535.2319870-7-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé
94a4f8167d gitlab-ci: Split CONFIGURE_ARGS one argument per line for build-disabled
We will keep adding/removing options to our 'configure' script,
so for easier maintainability it makes sense to have CONFIGURE_ARGS
declared as one option per line. This way we can review diff easily
(or rebase/cherry-pick).

No logical change.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201108204535.2319870-6-philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé
37b6fd6658 gitlab-ci: Document 'build-tcg-disabled' is a KVM X86 job
Document what this job cover (build X86 targets with
KVM being the single accelerator available).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-2-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-12-09 08:04:34 +01:00
Philippe Mathieu-Daudé
1352d5688d gitlab-ci: Move trace backend tests across to gitlab
Similarly to commit 8cdb2cef3f, move the trace backend
tests to GitLab.

Note the User-Space Tracer backend is still tested on
Ubuntu by the s390x jobs on Travis-CI.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201111121234.3246812-3-philmd@redhat.com>
Message-Id: <20201117173635.29101-7-alex.bennee@linaro.org>
2020-11-23 09:55:23 +00:00
Alex Bennée
ac74e282d4 gitlab: move remaining x86 check-tcg targets to gitlab
The GCC check-tcg (user) test in particular was very prone to timing
out on Travis. We only actually need to move the some-softmmu builds
across as we already have coverage for linux-user.

As --enable-debug-tcg does increase the run time somewhat as more
debug is put in let's restrict that to just the plugins build. It's
unlikely that a plugins enabled build is going to hide a sanity
failure in core TCG code so let the plugin builds do the heavy lifting
on checking TCG sanity so the non-plugin builds can run swiftly.

Now the only remaining check-tcg builds on Travis are for the various
non-x86 arches.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201117173635.29101-5-alex.bennee@linaro.org>
2020-11-23 09:52:57 +00:00
Daniel P. Berrangé
d0f26e68a0 gitlab: force enable docs build in Fedora, Ubuntu, Debian
Meson runs a test to see if Sphinx works, and automatically disables it
on error. This can lead to the CI jobs skipping docs build without
maintainers noticing the problem. Use --enable-docs to force a fatal
error if Sphinx doesn't work on the jobs where we expect it to be OK.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20201102130926.161183-3-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-10 08:51:30 +01:00
Daniel P. Berrangé
4daa9055be gitlab: publish the docs built during CI
Most of the build jobs will create the sphinx documentation. If we
expose this as an artifact of a "pages" job in a "public" directory, it
will get published using GitLab Pages. This means a user can push a
branch with docs changes to GitLab and view the results at

  https://yourusername.gitlab.io/qemu/

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20201102130926.161183-2-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-10 08:51:30 +01:00
Philippe Mathieu-Daudé
7433a6860b gitlab-ci: Drop generic cache rule
This cache rule is meant for Avocado artifacts, but affects
all jobs. Moreover the 'acceptance_template' template already
include a more detailled rule to cache artifacts.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201108221925.2344515-2-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-09 18:34:21 +01:00
Philippe Mathieu-Daudé
8d77ce1d89 gitlab-ci: Clone from GitLab itself
Let GitLab runners use GitLab repository directly.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201022123302.2884788-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2020-10-27 09:53:51 +00:00
Thomas Huth
67202baeaa tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
The tests are running in containers here, so it should be OK to
run with AVOCADO_ALLOW_UNTRUSTED_CODE enabled in this case.

Message-Id: <20201023073351.251332-4-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Paolo Bonzini
c47110d90f ci: include configure and meson logs in all jobs if configure fails
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:53 -04:00
Peter Maydell
b37da83763 * qtest improvements (test for crash found with the fuzzer, increase
downtime in migration test, less verbose output when running w/o KVM)
 * Improve handling of acceptance tests in the Gitlab-CI
 * Run checkpatch.pl in the Gitlab-CI
 * Improve the gitlab-pipeline-status script
 * Misc patches (mark 'moxie' as deprecated, remove stale .gitignore files, ...)
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl+FhiIRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbXfcBAAkMc4eUbZ0Wkm7M7TdIRkn1vQEstgvyJN
 6t02MuqY0R01rdbIBAnCLSw9okxfCTf7Q33VmC7snLtPo6WmvYIPAXZAnUiz13K1
 hGhMJfEY0JSyPEXlENMC/SWcRfNuHud6OPp6KePvn6EQsVZ5CR9SeO5zMsCVj2SP
 bMaBYIAJsVCEHkR2lq9UXbjckjyO0GQnQ/oR3mNiqDLYBmrXUOxIFMBctgfbuUtm
 uPuvvknHVQa8foD18qVJ8QYZrpwrqN4edFjcoW3yvwfX6OOhTnx+pY43BG/of9YB
 OoRY7V4VN8aYmVR08sqyn6PRNpXW9WcSUn8D3JNeiAhLzO/8H197JhHwFVvbZc7t
 puLECIINy91wH2i3Onx7HWhss3XLUK3HsvWNLrvLui6vdbFHEtiW2/0GbwJzrcA0
 a9inH7bvI7BlPiIau/J7goaDv0fzZ7xVXlQcrM8hC9oCWH5gvmvcgTBWJn/5OxUZ
 fov3iFxcRWslFSQe+D66gBceIl/fScF+TUmPoWyeSlD/f1OR2WW+q8N1FvnbLflz
 oPutIoja8b6CobzAzp8Igc6/9uQvzCAFB92Y8q1Og7eguQybw7dDtbArjBmjUBVi
 slFWoY8/ri2+uyiPsyU13Yfu9N5myqdwIQeM7H8sQ7qS40QHp0z2tj18o951xH2w
 WJv3PlGcez4=
 =lCRK
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-10-13' into staging

* qtest improvements (test for crash found with the fuzzer, increase
  downtime in migration test, less verbose output when running w/o KVM)
* Improve handling of acceptance tests in the Gitlab-CI
* Run checkpatch.pl in the Gitlab-CI
* Improve the gitlab-pipeline-status script
* Misc patches (mark 'moxie' as deprecated, remove stale .gitignore files, ...)

# gpg: Signature made Tue 13 Oct 2020 11:49:06 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-10-13: (23 commits)
  scripts/ci/gitlab-pipeline-status: wait for pipeline creation
  scripts/ci/gitlab-pipeline-status: use more descriptive exceptions
  scripts/ci/gitlab-pipeline-status: handle keyboard interrupts
  scripts/ci/gitlab-pipeline-status: refactor parser creation
  scripts/ci/gitlab-pipeline-status: give early feedback on running pipelines
  scripts/ci/gitlab-pipeline-status: improve message regarding timeout
  scripts/ci/gitlab-pipeline-status: make branch name configurable
  gitlab: assign python helper files to GitLab maintainers section
  gitlab: add a CI job to validate the DCO sign off
  gitlab: add a CI job for running checkpatch.pl
  configure: fixes indent of $meson setup
  docs/system/deprecated: Mark the 'moxie' CPU as deprecated
  Remove superfluous .gitignore files
  MAINTAINERS: Ignore bios-tables-test in the qtest section
  Add a comment in bios-tables-test.c to clarify the reason behind approach
  softmmu/vl: Be less verbose about missing KVM when running the qtests
  tests/migration: Allow longer timeouts
  qtest: add fuzz test case
  Acceptance tests: show test report on GitLab CI
  Acceptance tests: do not show canceled test logs on GitLab CI
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-13 12:46:26 +01:00