xemu/ui
Li Qiang 6bf21f3d83 vnc: fix memory leak when vnc disconnect
Currently when qemu receives a vnc connect, it creates a 'VncState' to
represent this connection. In 'vnc_worker_thread_loop' it creates a
local 'VncState'. The connection 'VcnState' and local 'VncState' exchange
data in 'vnc_async_encoding_start' and 'vnc_async_encoding_end'.
In 'zrle_compress_data' it calls 'deflateInit2' to allocate the libz library
opaque data. The 'VncState' used in 'zrle_compress_data' is the local
'VncState'. In 'vnc_zrle_clear' it calls 'deflateEnd' to free the libz
library opaque data. The 'VncState' used in 'vnc_zrle_clear' is the connection
'VncState'. In currently implementation there will be a memory leak when the
vnc disconnect. Following is the asan output backtrack:

Direct leak of 29760 byte(s) in 5 object(s) allocated from:
    0 0xffffa67ef3c3 in __interceptor_calloc (/lib64/libasan.so.4+0xd33c3)
    1 0xffffa65071cb in g_malloc0 (/lib64/libglib-2.0.so.0+0x571cb)
    2 0xffffa5e968f7 in deflateInit2_ (/lib64/libz.so.1+0x78f7)
    3 0xaaaacec58613 in zrle_compress_data ui/vnc-enc-zrle.c:87
    4 0xaaaacec58613 in zrle_send_framebuffer_update ui/vnc-enc-zrle.c:344
    5 0xaaaacec34e77 in vnc_send_framebuffer_update ui/vnc.c:919
    6 0xaaaacec5e023 in vnc_worker_thread_loop ui/vnc-jobs.c:271
    7 0xaaaacec5e5e7 in vnc_worker_thread ui/vnc-jobs.c:340
    8 0xaaaacee4d3c3 in qemu_thread_start util/qemu-thread-posix.c:502
    9 0xffffa544e8bb in start_thread (/lib64/libpthread.so.0+0x78bb)
    10 0xffffa53965cb in thread_start (/lib64/libc.so.6+0xd55cb)

This is because the opaque allocated in 'deflateInit2' is not freed in
'deflateEnd'. The reason is that the 'deflateEnd' calls 'deflateStateCheck'
and in the latter will check whether 's->strm != strm'(libz's data structure).
This check will be true so in 'deflateEnd' it just return 'Z_STREAM_ERROR' and
not free the data allocated in 'deflateInit2'.

The reason this happens is that the 'VncState' contains the whole 'VncZrle',
so when calling 'deflateInit2', the 's->strm' will be the local address.
So 's->strm != strm' will be true.

To fix this issue, we need to make 'zrle' of 'VncState' to be a pointer.
Then the connection 'VncState' and local 'VncState' exchange mechanism will
work as expection. The 'tight' of 'VncState' has the same issue, let's also turn
it to a pointer.

Reported-by: Ying Fang <fangying1@huawei.com>
Signed-off-by: Li Qiang <liq3ea@163.com>
Message-id: 20190831153922.121308-1-liq3ea@163.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-09-17 13:45:10 +02:00
..
icons ui: install logo icons to $prefix/share/icons 2019-01-21 09:43:13 +01:00
keycodemapdb@6b3d716e2b ui: update keycodemapdb to get py3 fixes 2018-02-05 19:53:55 -02:00
shader opengl: add flipping vertex shader 2017-10-17 10:25:42 +02:00
cocoa.m hw/core: Move cpu.c, cpu.h from qom/ to hw/core/ 2019-08-21 13:24:01 +02:00
console-gl.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
console.c console: fix cell overflow 2019-07-03 10:57:12 +02:00
curses_keys.h curses: support wide input 2019-03-11 08:39:02 +01:00
curses.c curses: assert get_wch return value is okay 2019-08-21 11:19:40 +02:00
cursor_hidden.xpm
cursor_left_ptr.xpm
cursor.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
egl-context.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
egl-headless.c ui/egl: fix framebuffer reads 2019-09-17 13:42:02 +02:00
egl-helpers.c ui/egl: fix framebuffer reads 2019-09-17 13:42:02 +02:00
gtk-egl.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
gtk-gl-area.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
gtk.c sysemu: Split sysemu/runstate.h off sysemu/sysemu.h 2019-08-16 13:37:36 +02:00
input-barrier.c ui: add an embedded Barrier client 2019-09-17 13:43:22 +02:00
input-barrier.h ui: add an embedded Barrier client 2019-09-17 13:43:22 +02:00
input-keymap.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
input-legacy.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
input-linux.c input-linux: add shift+shift as a grab toggle 2019-08-21 12:25:46 +02:00
input.c sysemu: Split sysemu/runstate.h off sysemu/sysemu.h 2019-08-16 13:37:36 +02:00
kbd-state.c Include qemu/queue.h slightly less 2019-08-16 13:31:52 +02:00
keymaps.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
keymaps.h Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
Makefile.objs ui: add an embedded Barrier client 2019-09-17 13:43:22 +02:00
qemu-pixman.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
qemu-x509.h
qemu.desktop ui: fix icon display for GTK frontend under GNOME Shell with Wayland 2019-01-21 09:43:13 +01:00
sdl2-2d.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
sdl2-gl.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
sdl2-input.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
sdl2.c sysemu: Split sysemu/runstate.h off sysemu/sysemu.h 2019-08-16 13:37:36 +02:00
shader.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
spice-app.c Include qemu/module.h where needed, drop it from qemu-common.h 2019-06-12 13:18:33 +02:00
spice-core.c sysemu: Split sysemu/runstate.h off sysemu/sysemu.h 2019-08-16 13:37:36 +02:00
spice-display.c Clean up inclusion of sysemu/sysemu.h 2019-08-16 13:31:53 +02:00
spice-input.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
trace-events trace-events: Fix attribution of trace points to source 2019-03-22 16:18:07 +00:00
vgafont.h
vnc_keysym.h ui: add next and prior keysyms 2017-07-27 14:23:09 +02:00
vnc-auth-sasl.c authz: delete existing ACL implementation 2019-02-26 15:32:19 +00:00
vnc-auth-sasl.h Include qemu/main-loop.h less 2019-08-16 13:31:52 +02:00
vnc-auth-vencrypt.c authz: delete existing ACL implementation 2019-02-26 15:32:19 +00:00
vnc-auth-vencrypt.h
vnc-enc-hextile-template.h ui: vnc: finish removing TABs 2019-02-05 16:50:18 +01:00
vnc-enc-hextile.c
vnc-enc-tight.c vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
vnc-enc-tight.h
vnc-enc-zlib.c vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
vnc-enc-zrle.c vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
vnc-enc-zrle.h
vnc-enc-zrle.inc.c vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
vnc-enc-zywrle-template.c avoid TABs in files that only contain a few 2019-01-11 15:46:56 +01:00
vnc-enc-zywrle.h ui: vnc: finish removing TABs 2019-02-05 16:50:18 +01:00
vnc-jobs.c vnc: fix memleak of the "vnc-worker-output" name 2018-08-24 08:40:10 +02:00
vnc-jobs.h ui/vnc: Drop unused vnc_has_job() and vnc_jobs_clear() 2017-02-08 14:59:36 +01:00
vnc-palette.c
vnc-palette.h Include qapi/qmp/qlist.h exactly where needed 2018-02-09 13:52:15 +01:00
vnc-stubs.c vnc: add qapi/error.h include to stubs 2018-02-16 12:23:21 +01:00
vnc-ws.c authz: delete existing ACL implementation 2019-02-26 15:32:19 +00:00
vnc-ws.h
vnc.c vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
vnc.h vnc: fix memory leak when vnc disconnect 2019-09-17 13:45:10 +02:00
x_keymap.c ui: fix keymap detection under Xwayland 2018-04-10 11:21:54 +02:00
x_keymap.h ui: convert GTK and SDL1 frontends to keycodemapdb 2018-01-25 15:02:00 +01:00