xemu/include/sysemu
Stefan Hajnoczi 2362a28ea1 iothread: fix iothread_stop() race condition
There is a small chance that iothread_stop() hangs as follows:

  Thread 3 (Thread 0x7f63eba5f700 (LWP 16105)):
  #0  0x00007f64012c09b6 in ppoll () at /lib64/libc.so.6
  #1  0x000055959992eac9 in ppoll (__ss=0x0, __timeout=0x0, __nfds=<optimized out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:77
  #2  0x000055959992eac9 in qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>) at util/qemu-timer.c:322
  #3  0x0000559599930711 in aio_poll (ctx=0x55959bdb83c0, blocking=blocking@entry=true) at util/aio-posix.c:629
  #4  0x00005595996806fe in iothread_run (opaque=0x55959bd78400) at iothread.c:59
  #5  0x00007f640159f609 in start_thread () at /lib64/libpthread.so.0
  #6  0x00007f64012cce6f in clone () at /lib64/libc.so.6

  Thread 1 (Thread 0x7f640b45b280 (LWP 16103)):
  #0  0x00007f64015a0b6d in pthread_join () at /lib64/libpthread.so.0
  #1  0x00005595999332ef in qemu_thread_join (thread=<optimized out>) at util/qemu-thread-posix.c:547
  #2  0x00005595996808ae in iothread_stop (iothread=<optimized out>) at iothread.c:91
  #3  0x000055959968094d in iothread_stop_iter (object=<optimized out>, opaque=<optimized out>) at iothread.c:102
  #4  0x0000559599857d97 in do_object_child_foreach (obj=obj@entry=0x55959bdb8100, fn=fn@entry=0x559599680930 <iothread_stop_iter>, opaque=opaque@entry=0x0, recurse=recurse@entry=false) at qom/object.c:852
  #5  0x0000559599859477 in object_child_foreach (obj=obj@entry=0x55959bdb8100, fn=fn@entry=0x559599680930 <iothread_stop_iter>, opaque=opaque@entry=0x0) at qom/object.c:867
  #6  0x0000559599680a6e in iothread_stop_all () at iothread.c:341
  #7  0x000055959955b1d5 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4913

The relevant code from iothread_run() is:

  while (!atomic_read(&iothread->stopping)) {
      aio_poll(iothread->ctx, true);

and iothread_stop():

  iothread->stopping = true;
  aio_notify(iothread->ctx);
  ...
  qemu_thread_join(&iothread->thread);

The following scenario can occur:

1. IOThread:
  while (!atomic_read(&iothread->stopping)) -> stopping=false

2. Main loop:
  iothread->stopping = true;
  aio_notify(iothread->ctx);

3. IOThread:
  aio_poll(iothread->ctx, true); -> hang

The bug is explained by the AioContext->notify_me doc comments:

  "If this field is 0, everything (file descriptors, bottom halves,
  timers) will be re-evaluated before the next blocking poll(), thus the
  event_notifier_set call can be skipped."

The problem is that "everything" does not include checking
iothread->stopping.  This means iothread_run() will block in aio_poll()
if aio_notify() was called just before aio_poll().

This patch fixes the hang by replacing aio_notify() with
aio_bh_schedule_oneshot().  This makes aio_poll() or g_main_loop_run()
to return.

Implementing this properly required a new bool running flag.  The new
flag prevents races that are tricky if we try to use iothread->stopping.
Now iothread->stopping is purely for iothread_stop() and
iothread->running is purely for the iothread_run() thread.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20171207201320.19284-6-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-12-19 10:25:09 +00:00
..
accel.h vl: convert -tb-size to qemu_strtoul 2017-07-04 14:39:28 +02:00
arch_init.h audio: Move arch_init audio code to hw/audio/soundhw.c 2017-05-19 10:48:53 +02:00
balloon.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
block-backend.h block: move ThrottleGroup membership to ThrottleGroupMember 2017-09-05 16:47:51 +02:00
blockdev.h blockdev: Make orphaned -drive fatal 2017-02-21 13:17:45 +01:00
bt.h Clean up header guards that don't match their file name 2016-07-12 16:19:16 +02:00
cpus.h migration: Mark CPU states dirty before incoming migration/loadvm 2017-06-06 08:53:24 +10:00
cryptodev.h cryptodev: wrap the ready flag 2017-01-10 07:02:52 +02:00
device_tree.h Clean up ill-advised or unusual header guards 2016-07-12 16:20:46 +02:00
dma.h dma-helpers: explicitly pass alignment into DMA helpers 2016-10-27 16:29:13 -04:00
dump-arch.h dump: allow target to set the physical base 2016-01-15 14:40:25 +00:00
dump.h dump: add guest ELF note 2017-10-15 05:54:40 +03:00
hax.h i386/hax: remove duplicated includes 2017-12-18 17:07:02 +03:00
hostmem.h hostmem: introduce host_memory_backend_mr_inited() 2017-04-20 15:22:41 -03:00
hw_accel.h migration: Mark CPU states dirty before incoming migration/loadvm 2017-06-06 08:53:24 +10:00
iothread.h iothread: fix iothread_stop() race condition 2017-12-19 10:25:09 +00:00
kvm_int.h kvm-all: add support for multiple address spaces 2015-07-06 17:59:43 +02:00
kvm.h kvm: check KVM_CAP_SYNC_MMU with kvm_vm_check_extension() 2017-10-02 14:38:06 +02:00
memory_mapping.h Use scripts/clean-includes to drop redundant qemu/typedefs.h 2016-03-22 22:20:16 +01:00
numa.h spapr: replace numa_get_node() with lookup in pc-dimm list 2017-12-15 09:49:24 +11:00
os-posix.h util: drop old utimensat() compat code 2017-05-25 10:30:14 +02:00
os-win32.h log: Add locking to large logging blocks 2016-11-01 10:29:03 -06:00
qtest.h include: Clean up includes 2016-02-23 12:43:05 +00:00
replay.h shutdown: Preserve shutdown cause through replay 2017-05-23 13:28:17 +02:00
reset.h hw: move reset handlers from vl.c to hw/core 2017-01-16 17:52:35 +01:00
rng-random.h rng-random: rename RndRandom to RngRandom 2016-05-23 12:18:43 +05:30
rng.h rng: switch request queue to QSIMPLEQ 2016-03-08 12:54:14 +05:30
seccomp.h seccomp: Don't include libseccomp from QEMU header 2017-09-22 09:48:33 +01:00
sysemu.h hmp-commands: Remove the deprecated usb_add and usb_del 2017-12-14 10:16:52 +00:00
tpm_backend.h tpm: tpm_emulator: get and set buffer size of device 2017-12-14 23:39:15 -05:00
tpm.h acpi: change TPM TIS data conditions 2017-12-14 23:39:15 -05:00
watchdog.h watchdog.h: Drop local redefinition of actions enum 2017-10-02 08:41:03 +02:00
xen-mapcache.h xen/mapcache: introduce xen_replace_cache_entry() 2017-07-18 14:16:09 -07:00