Commit Graph

77307 Commits

Author SHA1 Message Date
Pavel Dovgalyuk
1c80c87c8c tests/acceptance: refactor boot_linux to allow code reuse
This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073593167.20809.17582679291556188984.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Pavel Dovgalyuk
12121c496f tests/acceptance: refactor boot_linux_console test to allow code reuse
This patch splits code in BootLinuxConsole class into two different
classes to allow reusing it by record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073588490.20809.13942096070255577558.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Pavel Dovgalyuk
a5ba86d423 tests/acceptance: allow console interaction with specific VMs
Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073587933.20809.5122618715976660635.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Dr. David Alan Gilbert
2c9120a223 tests/acceptance/migration.py: Wait for both sides
When the source finishes migration the destination will still be
receiving the data sent by the source, so it might not have quite
finished yet, so won't quite have reached 'completed'.
This lead to occasional asserts in the next few checks.

After the source has finished, check the destination as well.
(We can't just switch to checking the destination, because it doesn't
give a status until it has started receiving the migration).

Reported-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528112404.121972-1-dgilbert@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Philippe Mathieu-Daudé
83389e22c5 tests/migration/guestperf: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-7-philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
6ee982c9ab tests/vm: allow wait_ssh() to specify command
This allows for waiting for completion of arbitrary commands.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-7-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
e56c45047b tests/vm: Add ability to select QEMU from current build
Added a new special variable QEMU_LOCAL=1, which
will indicate to take the QEMU binary from the current
build.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-6-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Robert Foley
d5326a2437 tests/vm: Pass --debug through for vm-boot-ssh
This helps debug issues that occur during the boot sequence.

Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200529203458.1038-5-robert.foley@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
c95dddce49 python/qemu/qtest: Check before accessing _qtest
It can be None; so add assertions or exceptions where appropriate to
guard the access accordingly.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-30-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
de210ec53c python/qemu/qmp: assert sockfile is not None
In truth, if you don't do this, you'll just get a TypeError
exception. Now, you'll get an AssertionError.

Is this tangibly better? No.
Does mypy complain less? Yes.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-21-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
7af67d694e python/qemu/qmp: use True/False for non/blocking modes
The type system doesn't want integers.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-15-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
1dda0404d8 python/qemu: Adjust traceback typing
mypy considers it incorrect to use `bool` to statically return false,
because it will assume that it could conceivably return True, and gives
different analysis in that case. Use a None return to achieve the same
effect, but make mypy happy.

Note: Pylint considers function signatures as code that might trip the
duplicate-code checker. I'd rather not disable this as it does not
trigger often in practice, so I'm disabling it as a one-off and filed a
change request; see https://github.com/PyCQA/pylint/issues/3619

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-14-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
0add048fbd python/qemu: fix socket.makefile() typing
Note:

A bug in typeshed (https://github.com/python/typeshed/issues/3977)
misinterprets the type of makefile(). Work around this by explicitly
stating that we are opening a text-mode file.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-13-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
3797dbcbb7 python/qemu: remove Python2 style super() calls
Use the Python3 style instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-12-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
John Snow
8dfac2edb2 python/qemu: delint; add flake8 config
Mostly, ignore the "no bare except" rule, because flake8 is not
contextual and cannot determine if we re-raise. Pylint can, though, so
always prefer pylint for that.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-5-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
John Snow
9b8ccd6d5b python/qemu: delint and add pylintrc
Bring our these files up to speed with pylint 2.5.0.
Add a pylintrc file to formalize which pylint subset
we are targeting.

The similarity ignore is there to suppress similarity
reports across imports, which for typing constants,
are going to trigger this report erroneously.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200528222129.23826-4-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
John Snow
053774bdec python/qemu/machine: remove logging configuration
Python 3.5 and above do not print a warning when logging is not
configured. As a library, it's best practice to leave logging
configuration to the client executable.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514055403.18902-22-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:30 +02:00
Vladimir Sementsov-Ogievskiy
e0e925a611 python/qemu/machine: add kill() method
Add method to hard-kill vm, without any quit commands.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <20200217150246.29180-19-vsementsov@virtuozzo.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
John Snow
2d110c1149 python: remove more instances of sys.version_info
We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514035230.25756-1-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
John Snow
c7b942d7f8 scripts/qmp: Fix shebang and imports
There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20200528222129.23826-2-jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
5aa628045d scripts/modules/module_block: Use Python 3 interpreter & add pseudo-main
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-6-philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
e57a707a82 scripts/kvm/vmxcap: Use Python 3 interpreter and add pseudo-main()
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-5-philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
06d4c71f48 scripts/qmp: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
806c200ef4 scripts/qemu-gdb: Use Python 3 interpreter
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
6acc5c0469 scripts/qemugdb: Remove shebang header
These scripts are loaded as plugin by GDB (and they don't
have any __main__ entry point). Remove the shebang header.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200512103238.7078-2-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 13:56:46 +02:00
Philippe Mathieu-Daudé
b809667808 hw/m68k/mcf52xx: Replace hw_error() by qemu_log_mask()
hw_error() calls exit(). This a bit overkill when we can log
the accesses as unimplemented or guest error.

When fuzzing the devices, we don't want the whole process to
exit. Replace some hw_error() calls by qemu_log_mask().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <huth@tuxfamily.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>
2020-05-30 09:17:46 +02:00
Philippe Mathieu-Daudé
ccff1ae4df hw/m68k/mcf5206: Reduce m5206_mbar_read/write() offset arg to 16-bit
All calls to m5206_mbar_read/m5206_mbar_write are used with
'offset = hwaddr & 0x3ff', so we are sure the offset fits
in 16-bit.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200526094052.1723-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <huth@tuxfamily.org>
2020-05-30 09:16:28 +02:00
Peter Maydell
ce20db593f bitmaps patches for 2020-05-26
- fix non-blockdev migration of bitmaps when mirror job is in use
 - add bitmap sizing to 'qemu-img measure'
 - add 'qemu-img convert --bitmaps'
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAl7QAA8ACgkQp6FrSiUn
 Q2pRxgf+N+X929X+o6kELEFBXMckOn8LC94A/6RjZKYywY3nV1ynESYflwznbYqx
 xQzXA2fTHK0O9n9av9pSMqs612HvxQyCJ51btit9QyOKW79//5QiNBkmVzXRwaN7
 hNnq5eHgDtH7O0jJPbJ1kxY6oufaudl/npEbBINlXknkIyvtChRfMhkAEOcV0N/y
 OxnmFm02r3TENoDmLdCmmi6iSDBvUl2th4p5ICZ0D9qUOnL83NNnMoYNF1goH25Z
 AlJWLe577mV5rw3w+HpoPutJZffPPiveCgW04WaEOhF4b0RzJFYazw9DM7PQiC+T
 hDzL3m15ZmgocgDLbbXEJiCVsDUwTA==
 =83Zm
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging

bitmaps patches for 2020-05-26

- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'

# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
  iotests: Add test 291 to for qemu-img bitmap coverage
  qemu-img: Add convert --bitmaps option
  qemu-img: Factor out code for merging bitmaps
  qcow2: Expose bitmaps' size during measure
  iotests: Fix test 178
  migration: forbid bitmap migration by generated node-name
  migration: add_bitmaps_to_list: check disk name once
  iotests: 194: test also migration of dirty bitmap
  migration: fix bitmaps pre-blockdev migration with mirror job
  block/dirty-bitmap: add bdrv_has_named_bitmaps helper
  migration: refactor init_dirty_bitmap_migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-29 19:25:54 +01:00
Peter Maydell
c86274bc2e Testing and one plugin fix:
- support alternates for genisoimage to test/vm
   - add clang++ to clang tests
   - fix record/replay smoke test
   - enable more softfloat tests
   - better detection of hung gdb
   - upgrade aarch64 tcg test x-compile to gcc-10
   - fix plugin cpu_index clash vs threads
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl7OazAACgkQ+9DbCVqe
 KkRFQwf/VPIy5u556hyXsXLW4cZeJG8zx4fOUhkLTecDL1ch6A+9io1EIyWCmkyU
 GmyuR7Lg5J3f5gsOHydNTiOKu3/VKYkjFAGhGGjz2jDHFz7tA3UWgMO58Nce0g5R
 IxNGD0IrJwyM0m1H5tB8NEkd7AAOnpGslr0YoE5eX1MHpeXSiDmXxIvYUA/7nktY
 AFld8wNh9IG1Gg8Ww5Iyxn4fuxggEtzFFvvjrZVZq5rKdDgv+9D3M2cIjNvfmSGV
 zIBjZBOFwopGk9cNMTk91oo4mEqH3BBrCHsxAmOnyf4WzH4us8TdRCr9dHKzxiFt
 YN/3iOP2IUZqstGGV/PY50WI0RQEoQ==
 =2Krq
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1' into staging

Testing and one plugin fix:

  - support alternates for genisoimage to test/vm
  - add clang++ to clang tests
  - fix record/replay smoke test
  - enable more softfloat tests
  - better detection of hung gdb
  - upgrade aarch64 tcg test x-compile to gcc-10
  - fix plugin cpu_index clash vs threads

# gpg: Signature made Wed 27 May 2020 14:29:20 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1:
  tests/tcg: add new threadcount test
  linux-user: properly "unrealize" vCPU object
  cpus-common: ensure auto-assigned cpu_indexes don't clash
  tests/docker: use a gcc-10 based image for arm64 tests
  tests/docker: add debian11 base image
  tests/tcg: better detect confused gdb which can't connect
  tests/fp: split and audit the conversion tests
  tests/fp: enable extf80_le_quite tests
  tests/tcg: fix invocation of the memory record/replay tests
  travis.yml: Use clang++ in the Clang tests
  tests/vm: pass --genisoimage to basevm script
  configure: add alternate binary for genisoimage

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-29 17:41:45 +01:00
Eric Blake
cf2d1203dc iotests: Add test 291 to for qemu-img bitmap coverage
Add a new test covering the 'qemu-img bitmap' subcommand, as well as
'qemu-img convert --bitmaps', both added in recent patches.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521192137.1120211-6-eblake@redhat.com>
2020-05-28 13:16:30 -05:00
Eric Blake
15e39ad950 qemu-img: Add convert --bitmaps option
Make it easier to copy all the persistent bitmaps of (the top layer
of) a source image along with its guest-visible contents, by adding a
boolean flag for use with qemu-img convert.  This is basically
shorthand, as the same effect could be accomplished with a series of
'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source'
commands, or by their corresponding QMP commands.

Note that this command will fail in the same scenarios where 'qemu-img
measure' omits a 'bitmaps size:' line, namely, when either the source
or the destination lacks persistent bitmap support altogether.

See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893

While touching this, clean up a couple coding issues spotted in the
same function: an extra blank line, and merging back-to-back 'if
(!skip_create)' blocks.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-5-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-28 13:16:30 -05:00
Eric Blake
6c729dd832 qemu-img: Factor out code for merging bitmaps
The next patch will add another client that wants to merge dirty
bitmaps; it will be easier to refactor the code to construct the QAPI
struct correctly into a helper function.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-4-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-28 13:16:30 -05:00
Eric Blake
5d72c68b49 qcow2: Expose bitmaps' size during measure
It's useful to know how much space can be occupied by qcow2 persistent
bitmaps, even though such metadata is unrelated to the guest-visible
data.  Report this value as an additional QMP field, present when
measuring an existing image and output format that both support
bitmaps.  Update iotest 178 and 190 to updated output, as well as new
coverage in 190 demonstrating non-zero values made possible with the
recently-added qemu-img bitmap command (see 3b51ab4b).

The new 'bitmaps size:' field is displayed automatically as part of
'qemu-img measure' any time it is present in QMP (that is, any time
both the source image being measured and destination format support
bitmaps, even if the measurement is 0 because there are no bitmaps
present).  If the field is absent, it means that no bitmaps can be
copied (source, destination, or both lack bitmaps, including when
measuring based on size rather than on a source image).  This behavior
is compatible with an upcoming patch adding 'qemu-img convert
--bitmaps': that command will fail in the same situations where this
patch omits the field.

The addition of a new field demonstrates why we should always
zero-initialize qapi C structs; while the qcow2 driver still fully
populates all fields, the raw and crypto drivers had to be tweaked to
avoid uninitialized data.

Consideration was also given towards having a 'qemu-img measure
--bitmaps' which errors out when bitmaps are not possible, and
otherwise sums the bitmaps into the existing allocation totals rather
than displaying as a separate field, as a potential convenience
factor.  But this was ultimately decided to be more complexity than
necessary when the QMP interface was sufficient enough with bitmaps
remaining a separate field.

See also: https://bugzilla.redhat.com/1779904

Reported-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521192137.1120211-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-28 13:16:16 -05:00
Eric Blake
ca01b7a641 iotests: Fix test 178
A recent change to qemu-img changed expected error message output, but
178 takes long enough to execute that it does not get run by 'make
check' or './check -g quick'.

Fixes: 43d589b074
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521192137.1120211-2-eblake@redhat.com>
2020-05-28 13:15:23 -05:00
Vladimir Sementsov-Ogievskiy
4ff5cc121b migration: forbid bitmap migration by generated node-name
It actually never worked with libvirt, as auto-generated names are
different on source and destination.

It's unsafe and useless to migrate by auto-generated node-names, so
let's forbid it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-7-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Vladimir Sementsov-Ogievskiy
82640edb88 migration: add_bitmaps_to_list: check disk name once
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-6-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Vladimir Sementsov-Ogievskiy
ae00aa2398 iotests: 194: test also migration of dirty bitmap
Test that dirty bitmap migration works when we deal with mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-5-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Vladimir Sementsov-Ogievskiy
107cfb7254 migration: fix bitmaps pre-blockdev migration with mirror job
Important thing for bitmap migration is to select destination block
node to obtain the migrated bitmap.

Prepatch, on source we use bdrv_get_device_or_node_name() to identify
the node, and on target we do bdrv_lookup_bs.
bdrv_get_device_or_node_name() returns blk name only for direct
children of blk. So, bitmaps of direct children of blks are migrated by
blk name and others - by node name.

Old libvirt is unprepared to bitmap migration by node-name,
node-names are mostly auto-generated. So actually only migration by blk
name works for it.

Newer libvirt will use new interface (which will be added soon) to
specify node-mapping for bitmaps migration explicitly. Still, let's
improve the current behavior a bit.

Now, consider classic libvirt migrations assisted by mirror block job:
mirror block job inserts filter, so our source is not a direct child of
blk, and bitmaps are migrated by node-names. And this just doesn't work
with auto-generated node names.

Let's fix it by using blk-name even if some implicit filters are
inserted.

Note2: we, of course, can't skip filters and use blk name to migrate
bitmaps in filtered node by blk name for this blk if these filters have
named bitmaps which should be migrated.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652424
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200521220648.3255-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: comment typo fix, shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Vladimir Sementsov-Ogievskiy
7ae89a0de9 block/dirty-bitmap: add bdrv_has_named_bitmaps helper
To be used for bitmap migration in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-3-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Vladimir Sementsov-Ogievskiy
38908bbc65 migration: refactor init_dirty_bitmap_migration
Split out handling one bs, it is needed for the following commit, which
will handle BlockBackends separately.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200521220648.3255-2-vsementsov@virtuozzo.com>
[eblake: shorter subject line]
Signed-off-by: Eric Blake <eblake@redhat.com>
2020-05-28 13:15:22 -05:00
Peter Maydell
b8bee16e94 hw/dispaly/sm501: bugfixes, add sanity checks.
hw/display: use tracepoints, misc cleanups.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABCgAGBQJez7AUAAoJEEy22O7T6HE4JvgQAKtszLGyFKlkFv1RDuUc35NY
 hWsEbglGPesn2jFMxL1TI7hr+yb7qyaakbsv1WBhRpmI22BXJcNRjGv9XlWDzp7i
 LnE2Zm5WwJ6Z53G70inybzIp1sRLvZXl8c0218/bCiH1i37OR+dwXGmg8lMhU13P
 QcVmzPYSVqj8/qkS6zmZHMbkVk2jonOp7S0wqxI7GHgAclB5K4BZHy4i+s0V2zYK
 KesIvVdyFvgqzIWvR3CwJC7HKVDBz4whDwzASNdICUbEj+C99wq4euZY7idsonoM
 NEbxIXK23iAKwl75hLiN9KDaXWTzo7wBpQze/6PDtz+l4n6zj0Ig9SRGIXAQ3+NW
 HjRDUBn0gXA0TTrSzP9F7/3nHX7khHjD87NxguFjiYRJCuEWNCIgqMkVb1Rgrag0
 WOlH+Rn0BJM0dedBRgdGvGnwvZouDpEdqimvQ0JD93PEun/x1yFy4t/U4EWI+RM7
 V9QiZ2kIgH/EbF4fkt7R+9Y86w5ZNn4YNbyP42zx5b/x0gcWzcU/yP/JPhsAP7gM
 EgzbTKW6KDqUQQtKn/7taHgpje/XK5AVtcqt7HbM46tF6koIUqTncSrEqnV7tAHy
 yagAqn4pQjs7m4Yd2Vk6jhVXM5x5vGqrRZUHQjroeX++vmetTjclNyMd6P9DIP+j
 +kCoG85UpPraCLHiQcu9
 =WVh1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20200528-pull-request' into staging

hw/dispaly/sm501: bugfixes, add sanity checks.
hw/display: use tracepoints, misc cleanups.

# gpg: Signature made Thu 28 May 2020 13:35:32 BST
# gpg:                using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20200528-pull-request: (21 commits)
  sm501: Remove obsolete changelog and todo comment
  sm501: Optimize small overlapping blits
  sm501: Replace hand written implementation with pixman where possible
  sm501: Clean up local variables in sm501_2d_operation
  sm501: Use BIT(x) macro to shorten constant
  sm501: Shorten long variable names in sm501_2d_operation
  sm501: Convert printf + abort to qemu_log_mask
  hw/display/pxa2xx_lcd: Replace printf() call by qemu_log_mask()
  hw/display/omap_dss: Replace fprintf() call by qemu_log_mask(LOG_UNIMP)
  hw/display/exynos4210_fimd: Use qemu_log_mask(GUEST_ERROR)
  hw/display/vmware_vga: Let the PCI device own its I/O MemoryRegion
  hw/display/vmware_vga: Replace printf() calls by qemu_log_mask(ERROR)
  hw/display/xlnx_dp: Replace disabled DPRINTF() by error_report()
  hw/display/dpcd: Convert debug printf()s to trace events
  hw/display/dpcd: Fix memory region size
  hw/display/cirrus_vga: Convert debug printf() to trace event
  hw/display/cirrus_vga: Use qemu_log_mask(ERROR) instead of debug printf
  hw/display/cirrus_vga: Use qemu_log_mask(UNIMP) instead of debug printf
  hw/display/cirrus_vga: Convert debug printf() to trace event
  hw/display/cg3: Convert debug printf()s to trace events
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-28 18:13:20 +01:00
Peter Maydell
a20ab81d22 Fixes and improvements for the gitlab-CI:
- Build with other images instead of the broken Debian containers
 - Fix building with the latest version of Clang (at least wrt. to
   the gitlab-CI pipeline)
 - Add Philippe, Alex and Wainer to the Gitlab-CI section in MAINTAINERS
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl7PgV8RHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbXnww/+OSn3x62FYTSmqbwfF/fTFnOqVndfKjkQ
 UhG4s6ttlFN2H+hgS9sxHB7cwk9WsyN/V7V8DMM3Ytdbmz9wsP8bLx+Q5wXHmIAl
 r/yk9AMrrJtlzGZnt1FcZMJdnrzky9VrqZgM1e0oL8mdPYvTiBmlpJqKoFk0tmWS
 WR0qKYFjETRCWtJgalMJTiYuRtkGIlTjXVo7vLmUNYKp+xCKP5Pn7J9JA+Ds+aXX
 /PuOztd9IT/T0K+F8lbt9iHVSAu+tsVn1CsZy9cLKEDwgmWFWrF2faXA8e9lZdKe
 NctTdClb+DILgezxdd9obIGBJKLy11xPXBopH+8cy9u5vWl1nNUu0tppni83bslB
 Kk4AB/qWLRFxz8xJ4U8jI3w62cH7u+gUIxiDeFuJxociLIhYQRL2otskiJ4q1RRD
 VOdX8PhsOa8LfV+LqJ+vfJQeYrliYaz/1wU+d6tFL3r+k9E+zI0Z+XCDa9z0pL0B
 aBh/UpsUUAUpLyy8JlYm6RejsnHchRqtx/1c+Bdsg84o4BQkFTrrLh3BZFQBncXp
 NYNXuWQi0Teotm01tvOHqZjqwE+2apKcKtdj0E6t0YDrn5Inbvj3/l64KeDEzwQN
 W0ppgJCJm/k5UkfkfEiFvwgvIn8jj3CrszVjlLZurIkDtTSrEqoNBv68AJewGPHL
 YCbuqk8Q8XE=
 =TThW
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-05-28' into staging

Fixes and improvements for the gitlab-CI:
- Build with other images instead of the broken Debian containers
- Fix building with the latest version of Clang (at least wrt. to
  the gitlab-CI pipeline)
- Add Philippe, Alex and Wainer to the Gitlab-CI section in MAINTAINERS

# gpg: Signature made Thu 28 May 2020 10:16:15 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-05-28:
  gitlab-ci: Determine the number of jobs dynamically
  gitlab-ci: Do not use the standard container images from gitlab
  gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder
  GitLab CI: avoid calling before_scripts on unintended jobs
  gitlab-ci: Remove flex/bison packages
  MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section
  linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-28 16:18:06 +01:00
Peter Maydell
87023dcc25 A single patch to avoid clashes with the regiser field macros.
-----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEE9sSsRtSTSGjTuM6PIeENKd+XcFQFAl7OsEcACgkQIeENKd+X
 cFSnDwgAmysoO9ERML98IWkxsGjakXhe3SCCij1FdwARYtQrJt/yjaUmg9tTWxa2
 qwrYDM70G7hNBMtqrJOGFG8nO8LWPkI6N88tNKrFO7ariO2uyb3mKdivt+EVjlhG
 o7vuhZ07xNmqote1zd1eeva+mu3a1pleL57dimlbkzINZp4Wc47n3TwT3WQNAO3J
 jKSbhqaFIXFpuLeXyFUNkd/liKFOcXXo0eV4XyKbYglcB2iX22m3NnGsH0PZMWoe
 hwKwOjk15fOeHbUcgkeah9vcZ2b5T17SmjU8ihHawU18CFaZ0zlAVWJdYWzdG6a/
 CFW2uQhLI6e7Y39PweEqvw7wyjxO3w==
 =X6Pz
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/alistair/tags/pull-register-api-20200527' into staging

A single patch to avoid clashes with the regiser field macros.

# gpg: Signature made Wed 27 May 2020 19:24:07 BST
# gpg:                using RSA key F6C4AC46D4934868D3B8CE8F21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>" [full]
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-register-api-20200527:
  hw/registerfields: Prefix local variables with underscore in macros

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-28 14:54:47 +01:00
Peter Maydell
aacc7c8be2 Error reporting patches for 2020-05-27
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl7N/xISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTdJYP/j0mnss0or9ZuchxNpFLTV1lBd0wuKqL
 79279BWsJ8AObmYPs0oXe1cfcq76NA7d6tVYrmz0pZ7BkYmXY0VgbBKeYXozcX5z
 6MJ+7+7wOJ1hO4oSTz9Jo1ZSpr8g0Be4m4DweHSyC9Uz6jCNAw3yJlPNsZ295jvk
 w2QD4ApwaofhF8SkNw5s4DE1piLpr8+IbGhY7YTKASQv3dIqTfjGSoz9MCKI4ZCA
 KHLLLkgV3NvCgxRKP2JclWUSsX6Er8joN2zUGlU1xBm6yTZqjWPLMxSysVkl+vcA
 1qx4GggDZ/zMcfyGBdXW8EAtT1QMfHU6zh9ED9toYbBohWFewTVCqkSaA2yVQZWx
 D69L6Q1KT/RIbrH6zEQaIEYcMmzvZWLy6EumKD9pgOAwDQlHv5QPT5hp6gonF/kI
 D8ZdT0jQ6JQqo42zi4J4hW4eH0r0Kf+79XsVz7wxe26YK762ZVnJLK/q7Fu1vb/u
 bBMJBYh/xdRO1GDE9JsomVq3mdXlz4ywQM+SFUiCj+peiO0C9JdDxE+PII1RaDqV
 wlJ7HW1eoKs4OG0gBjrJM2tn4q3BNpyr2BLSQONBSJEfWi6Z4OnimN45K5cEEJHj
 et2OCQetRt1zXmdrFdvX3Zn87PMZKfPmdaZrxzRvxk/VIP1f4vRgJ3D2FGP+E82H
 S/GWWRkw8pZy
 =b99a
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-error-2020-05-27' into staging

Error reporting patches for 2020-05-27

# gpg: Signature made Wed 27 May 2020 06:48:02 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-error-2020-05-27:
  i386: Fix x86_cpu_load_model() error API violation
  arm/sabrelite: Consistently use &error_fatal in sabrelite_init()
  mips/boston: Plug memory leak in boston_mach_init()
  mips/boston: Fix boston_mach_init() error handling
  mips/malta: Fix create_cps() error handling
  error: Use error_reportf_err() where appropriate
  tests/migration: Tighten error checking
  s390x/cpumodel: Fix harmless misuse of visit_check_struct()
  xen: Fix and improve handling of device_add usb-host errors
  nvdimm: Plug memory leak in uuid property setter

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-05-28 11:21:30 +01:00
BALATON Zoltan
fa0013a1bc sm501: Remove obsolete changelog and todo comment
Also update copyright year for latest changes

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 1392cad2ad1315a5a50409970e0af061821462e6.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00
BALATON Zoltan
fa70c2871f sm501: Optimize small overlapping blits
AmigaOS tends to do a lot of small blits (even 1 pixel). Avoid malloc
overhead by keeping around a buffer for this and only alloc when
blitting larger areas.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: 7946852258d528497e85f465327fc90b5c3b59fb.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00
BALATON Zoltan
b15a22bbcb sm501: Replace hand written implementation with pixman where possible
Besides being faster this should also prevent malicious guests to
abuse 2D engine to overwrite data or cause a crash.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: 58666389b6cae256e4e972a32c05cf8aa51bffc0.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00
BALATON Zoltan
3d0b096298 sm501: Clean up local variables in sm501_2d_operation
Make variables local to the block they are used in to make it clearer
which operation they are needed for.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: ae59f8138afe7f6a5a4a82539d0f61496a906b06.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00
BALATON Zoltan
2824809b7f sm501: Use BIT(x) macro to shorten constant
Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 124bf5de8d7cf503b32b377d0445029a76bfbd49.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00
BALATON Zoltan
6f8183b5dc sm501: Shorten long variable names in sm501_2d_operation
This increases readability and cleans up some confusing naming.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Message-id: b9b67b94c46e945252a73c77dfd117132c63c4fb.1590089984.git.balaton@eik.bme.hu
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-28 11:38:57 +02:00