Commit Graph

518 Commits

Author SHA1 Message Date
Michael Roth
a678e26cbe qapi: pad GenericList value fields to 64 bits
With the introduction of native list types, we now have types such as
int64List where the 'value' field is not a pointer, but the actual
64-bit value.

On 32-bit architectures, this can lead to situations where 'next' field
offset in GenericList does not correspond to the 'next' field in the
types that we cast to GenericList when using the visit_next_list()
interface, causing issues when we attempt to traverse linked list
structures of these types.

To fix this, pad the 'value' field of GenericList and other
schema-defined/native *List types out to 64-bits.

This is less memory-efficient for 32-bit architectures, but allows us to
continue to rely on list-handling interfaces that target GenericList to
simply visitor implementations.

In the future we can improve efficiency by defaulting to using native C
array backends to handle list of non-pointer types, which would be more
memory efficient in itself and allow us to roll back this change.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-30 09:08:38 -04:00
Stefan Hajnoczi
0412960617 tests: set MALLOC_PERTURB_ to expose memory bugs
glibc wipes malloc(3) memory when the MALLOC_PERTURB_ environment
variable is set.  The value of the environment variable determines the
bit pattern used to wipe memory.  For more information, see
http://udrepper.livejournal.com/11429.html.

Set MALLOC_PERTURB_ for gtester and qemu-iotests.  Note we pick a random
value from 1 to 255 to expose more bugs.  If you need to reproduce a
crash use 'show environment' in gdb to extract the MALLOC_PERTURB_
value from a core dump.

Both make check and qemu-iotests pass with MALLOC_PERTURB_ enabled.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1369661331-28041-1-git-send-email-stefanha@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-29 08:16:06 -05:00
Kevin Wolf
bd91ecbf5b qemu-iotests: Try creating huge qcow2 image
It's supposed to fail gracefully instead of segfaulting.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-24 16:17:55 +02:00
Kevin Wolf
c93331c914 qcow2.py: Subcommand for changing header fields
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-24 16:17:55 +02:00
Michael Roth
199e0f17f2 qapi: add native list coverage for QMP input visitor tests
This exercises schema-generated visitors for native list types and does
some sanity checking on validity of deserialized data.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:21 -04:00
Michael Roth
83c84667f5 qapi: add native list coverage for QMP output visitor tests
This exercises schema-generated visitors for native list types and does
some sanity checking on validity of serialized data.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:21 -04:00
Michael Roth
8addacddfe qapi: add native list coverage for visitor serialization tests
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:21 -04:00
Michael Roth
089f26bb73 qapi: fix visitor serialization tests for numbers/doubles
We never actually stored the stringified double values into the strings
before we did the comparisons. This left number/double values completely
uncovered in test-visitor-serialization tests.

Fixing this exposed a bug in our handling of large whole number values
in QEMU's JSON parser which is now fixed.

Simplify the code while we're at it by dropping the
calc_float_string_storage() craziness in favor of GStrings.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:21 -04:00
Michael Roth
e92cfa0d90 qapi: add QMP input test for large integers
Large integers previously got capped to LLONG_MAX/LLONG_MIN so we could
store them as int64_t. This could lead to silent errors occuring.

Now, we use a double to handle these cases.

Add a test to confirm that QMPInputVisitor handles this as expected if
we're expected an integer value: errors for out of range integer values
that got promoted to doubles in this fashion.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:21 -04:00
Anthony Liguori
95de21a430 Merge remote-tracking branch 'mjt/trivial-patches' into staging
# By Christophe Lyon (1) and others
# Via Michael Tokarev
* mjt/trivial-patches:
  target-moxie: replace target_phys_addr_t with hwaddr
  Rename hexdump to avoid FreeBSD libutil conflict
  remove some double-includes
  translate: remove redundantly included qemu/timer.h
  Remove twice include of qemu-common.h
  fix /proc/self/maps output

Message-id: 51977B44.1000302@msgid.tls.msk.ru
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-22 08:22:36 -05:00
Petar Jovanovic
0ba365f4a9 target-mips: fix EXTPDP and setting up pos field in the DSPControl reg
This change makes sure that modifications of pos field in the DSPControl
register do not trash other bits in the register. This bug can be triggered
with the additional test case in mips32-dsp/extpdp.c in this commit.

In addition to this, this change corrects incorrect calculation of the mask
for EXTPDP.

Signed-off-by: Petar Jovanovic <petar.jovanovic@imgtec.com>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2013-05-19 15:10:51 +02:00
Michael Tokarev
997aba8e25 remove some double-includes
Some source files #include the same header more than
once for no good reason.  Remove second #includes in
such cases.

Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2013-05-18 16:35:12 +04:00
Petar Jovanovic
489ed4bbae target-mips: fix incorrect behaviour for EXTP
The mask for EXTP instruction when size=31 has not been correctly
calculated.

The test (mips32-dsp/extp.c) has been extended to include the case that
triggers the issue.

Signed-off-by: Petar Jovanovic <petar.jovanovic@imgtec.com>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2013-05-17 19:29:40 +02:00
Anthony Liguori
6b41659f4e Merge remote-tracking branch 'luiz/queue/qmp' into staging
# By Michael Roth (1) and Zhangleiqiang (1)
# Via Luiz Capitulino
* luiz/queue/qmp:
  qapi: fix leak in unit tests
  qmp: fix handling of cmd with Equals in qmp-shell

Message-id: 1368625179-27962-1-git-send-email-lcapitulino@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-15 14:57:30 -05:00
Kevin Wolf
262f27b93f ide-test: Fix endianness problems
The test case passes on big endian hosts now (tested on ppc64)

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 1368622839-7084-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-15 10:31:15 -05:00
Michael Roth
ad7f375df6 qapi: fix leak in unit tests
qmp_output_get_qobject() increments the qobject's reference count. Since
we currently pass this straight into qobject_to_json() so we can feed
the data into a QMP input visitor, we never actually free the underlying
qobject when qmp_output_visitor_cleanup() is called. This causes leaks
on all of the QMP serialization tests.

Fix this by holding a pointer to the qobject and decref'ing it before
returning from qmp_deserialize().

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-15 08:58:43 -04:00
Anthony Liguori
cad2b59c19 Merge remote-tracking branch 'kwolf/for-anthony' into staging
# By Kevin Wolf (7) and Fam Zheng (3)
# Via Kevin Wolf
* kwolf/for-anthony:
  qemu-iotests: fix 017 018 for vmdk
  qemu-iotests: exclude vmdk and qcow from 043
  qemu-iotests: exclude vmdk for test 042
  qtest/ide-test: Test short and long PRDTs
  qtest/ide-test: Add simple DMA read/write test case
  qtest: Add IDE test case
  libqos/pci: Enable bus mastering
  ide: Reset BMIDEA bit when the bus master is stopped
  de_DE.po: Add missing leading spaces
  ahci: Don't allow creating slave drives

Message-id: 1368023344-29731-1-git-send-email-kwolf@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-08 15:54:36 -05:00
Petar Jovanovic
c0f5f9ce86 target-mips: fix incorrect behaviour for INSV
Corner case for INSV instruction when size=32 has not been correctly
implemented. The mask for size should be one bit wider, and preparing the
filter variable should be aware of this case too.

The test for INSV has been extended to include the case that triggers the
bug.

Signed-off-by: Petar Jovanovic <petar.jovanovic@imgtec.com>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2013-05-08 18:46:38 +02:00
Fam Zheng
794d00f71d qemu-iotests: fix 017 018 for vmdk
017 and 018 use /bin/mv to move base img from t.IMGFMG to t.IMGFMT.base
after filling data, this is not enough for vmdk, when t.IMGFMT is only a
description text file who points to t-{flat,s001,f001,...}.IMGFMT as
data extent, so testing such subformats alway fails on them.

This patch use the trick of temprorily changing TEST_IMG to avoid using
/bin/mv.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:50 +02:00
Fam Zheng
dbcdd7cb04 qemu-iotests: exclude vmdk and qcow from 043
043 tests recursive backing file by changing backing file. VMDK has not
implemented this yet, and qcow1 probably never will.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Fam Zheng
1afe272663 qemu-iotests: exclude vmdk for test 042
Zero sized disk is not supported by qemu vmdk driver, exclude vmdk from
the test script.

As tested on vmware-vdiskmanager and vmware workstation, zero sized disk
is not supported by vmware, either.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Kevin Wolf
948eaed171 qtest/ide-test: Test short and long PRDTs
This tests the behaviour of the DMA engine when the given PRDT contains
physical region descriptors for either more or less bytes than the
IDE request is for.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Kevin Wolf
b95739dcf5 qtest/ide-test: Add simple DMA read/write test case
This tests that single sectors can be successfully written and correctly
read back.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Kevin Wolf
acbe48013b qtest: Add IDE test case
This adds a simple IDE test case and starts by verifying that IDENTIFY
can be successfully used and return the correct serial number, version
and the WCE flag is set for cache=writeback.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Kevin Wolf
9f0332b8cf libqos/pci: Enable bus mastering
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-05-08 15:28:49 +02:00
Andreas Färber
cc9936a32f libqos: Relocate I2C files
Commit c4efe1cada (qtest: add libqos
including PCI support) created a libqos/ subdirectory but left the
existing I2C libqos files libi2c*.[hc] in tests/. Clean this up.

Signed-off-by: Andreas Färber <afaerber@suse.de>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Message-id: 1367502986-15104-1-git-send-email-afaerber@suse.de
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-05-03 12:03:14 -05:00
Anthony Liguori
25565e8595 Merge remote-tracking branch 'stefanha/block' into staging
# By Fam Zheng (8) and others
# Via Stefan Hajnoczi
* stefanha/block:
  qemu-iotests: Filter out 'adapter_type'
  nbd: support large NBD requests
  nbd: use g_slice_new() instead of a freelist
  qemu-iotests: Filter out vmdk creation options
  vmdk: add bdrv_co_write_zeroes
  vmdk: store fields of VmdkMetaData in cpu endian
  vmdk: change magic number to macro
  vmdk: Add option to create zeroed-grain image
  vmdk: add support for “zeroed‐grain” GTE
  vmdk: named return code.
  blockdev: Replace "undefined error" in qmp_block_resize
  block: add read-only support to VHDX image format.
  block: initial VHDX driver support framework - supports open and probe
  block: vhdx header for the QEMU support of VHDX images
  qemu: add castagnoli crc32c checksum algorithm
2013-05-03 11:20:02 -05:00
Fam Zheng
86abefd61e qemu-iotests: Filter out 'adapter_type'
Filter out vmdk creation option 'adapter_type' for vmdk. So that tests
with an explicit './check -o adapter_type=XXX' will not fail.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-05-03 13:06:22 +02:00
Petar Jovanovic
29851ee7c8 target-mips: fix calculation of overflow for SHLL.PH and SHLL.QB
This change corrects and simplifies how discard is calculated for shift
left logical vector instructions. It is used to detect overflow and set bit
22 in the DSPControl register.

The existing tests (shll_ph.c, shll_qb.c) are extended with the corner cases
that expose incorrectness in the previous implementation.

Signed-off-by: Petar Jovanovic <petar.jovanovic@imgtec.com>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2013-05-03 11:50:49 +02:00
Fam Zheng
50522d969b qemu-iotests: Filter out vmdk creation options
Cover new image creation options for vmdk, so we can use '-o
zeroed_grain=XXX' and '-o subformat=XXX' to run the tests successfully.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-05-03 10:34:16 +02:00
Jesse Larrew
beb54a87ba libqtest: only call fclose() on open files
libqtest.c can segfault when calling fclose() if the pidfile wasn't
opened successfully. This patch fixes the issue.

Signed-off-by: Jesse Larrew <jlarrew@linux.vnet.ibm.com>
Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1367250772-17928-1-git-send-email-jlarrew@linux.vnet.ibm.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-04-30 16:04:14 -05:00
Andreas Färber
1a63e059de libqos: Convert fw_cfg values to host endianness
The fw_cfg ABI is Little Endian, so byte-swap the generically read
byte array to host endianness.

This unbreaks the fw_cfg tests on ppc.

Signed-off-by: Andreas Färber <afaerber@suse.de>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Message-id: 1367167547-19931-1-git-send-email-afaerber@suse.de
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-04-29 08:28:15 -05:00
Kevin Wolf
8ec7d390b0 block: Disable driver-specific options for 1.5
We don't want to commit to the API yet before everything is worked out.
Disable it for the 1.5 release. This commit is meant to be reverted
after the 1.5 release.

The disabling of the driver-specific options is achieved by applying the
old checks while parsing the command line.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-26 13:26:28 +02:00
Stefan Hajnoczi
7da94ca741 qemu-iotests: add 053 unaligned compressed image size test
Test that qemu-img convert -c works when input image length is not a
multiple of the cluster size.

Previously an error message would be produced:

  qemu-img: error while compressing sector 0: Input/output error

Now that qcow2 and qcow handle this case the test passes successfully.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-04-22 11:37:12 +02:00
Kevin Wolf
2af5ef70af block: Fail gracefully when using a format driver on protocol level
Specifying the wrong driver could fail an assertion:

$ qemu-system-x86_64 -drive file.driver=qcow2,file=x
qemu-system-x86_64: block.c:721: bdrv_open_common: Assertion `file !=
((void *)0)' failed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2013-04-22 10:27:59 +02:00
Kevin Wolf
bdda92324d qemu-iotests: Fix _filter_qemu
$QEMU_PROG happens to be 'qemu' in my setup, so this sed command
replaces a bit too much. Restrict it to the start of the line and to
when it's followed by a colon, i.e. the form used by error messages.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-22 10:27:58 +02:00
Anthony Liguori
bf2a38d41e fw_cfg: add qtest test case
This validates some basic characteristics of fw_cfg.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-8-git-send-email-aliguori@us.ibm.com
2013-04-17 10:27:47 -05:00
Anthony Liguori
a875711af9 i440fx-test: add test for PAM functionality
This tests PAM settings for the i440fx.  This test does a lot of
byte MMIO which is fairly slow with qtest today.  But the test
does complete in under 2 seconds.

We don't fully emulate PAM largely because of limitations with
KVM so we #if 0 that part of the test case.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-7-git-send-email-aliguori@us.ibm.com
2013-04-17 10:27:47 -05:00
Anthony Liguori
9bda413c96 i440fx-test: add test to compare default register values
This test compares all of the default register values against the
spec.  It turns out we deviate in quite a few places.  These
places are really only visible to the BIOS though which is why
this hasn't created any problems.

The deviation actually happens in the core PCI layer so I suspect
it's not a simple fix if we really care to fix it.  For now, just
disable the affected checks.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-6-git-send-email-aliguori@us.ibm.com
2013-04-17 10:27:28 -05:00
Anthony Liguori
8a0743cf74 libqos: add malloc support
This is a very simple allocator for the PC platform.  It should
be possible to add backends for other platforms.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-5-git-send-email-aliguori@us.ibm.com
2013-04-17 10:26:05 -05:00
Anthony Liguori
234c69c5f9 libqos: add fw_cfg support
fw_cfg is needed to get the top of memory which is necessary for
doing PCI allocation and allocating RAM for DMA.

Add a PC version of fw_cfg and enough abstraction to support other
platforms.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-4-git-send-email-aliguori@us.ibm.com
2013-04-17 10:26:05 -05:00
Anthony Liguori
c4efe1cada qtest: add libqos including PCI support
This includes basic PCI support for the PC platform.  Enough
abstraction should be present to support non-PC platforms too.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-3-git-send-email-aliguori@us.ibm.com
2013-04-17 10:26:05 -05:00
Anthony Liguori
8a8fd63734 qtest: don't use system command to avoid double fork
Currently we waitpid on the child process we spawn off that does
nothing more than system() another process.  While this does not
appear to be incorrect, it's wasteful and confusing so get rid of
it.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Message-id: 1366123521-4330-2-git-send-email-aliguori@us.ibm.com
2013-04-17 10:26:04 -05:00
Petar Jovanovic
b1ca31d7ce target-mips: fix mipsdsp_mul_q15_q15 and tests for MAQ_SA_W_PHL/PHR
The operands for MAQ_SA_W.PHL/MAQ_SA_W.PHR must in specified format.
Otherwise, the results are unpredictable. Once the operands were corrected
in the tests (part of this change), a bug in mipsdsp_mul_q15_q15 became
visible.

This change corrects the tests for MAQ_SA_W.PHL/MAQ_SA_W.PHR and fixes
sign-related issue in mipsdsp_mul_q15_q15. It also removes unnecessary
comment.

Signed-off-by: Petar Jovanovic <petar.jovanovic@imgtec.com>
2013-04-15 16:07:57 +02:00
Richard W.M. Jones
342809e807 iotests: Add 'check -ssh' option to test Secure Shell block device.
Note in order to run these tests on ssh, you must be running a local
ssh daemon, and that daemon must accept loopback connections, and
ssh-agent has to be set up to allow logins on the local daemon.  In
other words, the following command should just work without demanding
any passphrase:

 ssh localhost

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-15 10:18:05 +02:00
Stefan Hajnoczi
c09b437b5f qemu-iotests: filter QEMU_PROG in 051.out
Filter the name of the QEMU executable so the output can be diffed no
matter what QEMU_PROG is (e.g. qemu-system-x86_64).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-15 10:18:05 +02:00
Kevin Wolf
47e5df2146 qemu-iotests: Add test for -drive options
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-15 08:26:18 +02:00
Kevin Wolf
cf07aecf95 qemu-iotests: A few more bdrv_pread/pwrite tests
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2013-04-15 08:26:18 +02:00
Markus Armbruster
e2ec3f9768 qjson: to_json() case QTYPE_QSTRING is buggy, rewrite
Known bugs in to_json():

* A start byte for a three-byte sequence followed by less than two
  continuation bytes is split into one-byte sequences.

* Start bytes for sequences longer than three bytes get misinterpreted
  as start bytes for three-byte sequences.  Continuation bytes beyond
  byte three become one-byte sequences.

  This means all characters outside the BMP are decoded incorrectly.

* One-byte sequences with the MSB are put into the JSON string
  verbatim when char is unsigned, producing invalid UTF-8.  When char
  is signed, they're replaced by "\\uFFFF" instead.

  This includes \xFE, \xFF, and stray continuation bytes.

* Overlong sequences are happily accepted, unless screwed up by the
  bugs above.

* Likewise, sequences encoding surrogate code points or noncharacters.

* Unlike other control characters, ASCII DEL is not escaped.  Except
  in overlong encodings.

My rewrite fixes them as follows:

* Malformed UTF-8 sequences are replaced.

  Except the overlong encoding \xC0\x80 of U+0000 is still accepted.
  Permits embedding NUL characters in C strings.  This trick is known
  as "Modified UTF-8".

* Sequences encoding code points beyond Unicode range are replaced.

* Sequences encoding code points beyond the BMP produce a surrogate
  pair.

* Sequences encoding surrogate code points are replaced.

* Sequences encoding noncharacters are replaced.

* ASCII DEL is now always escaped.

The replacement character is U+FFFD.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2013-04-13 19:40:25 +00:00
Markus Armbruster
1d50c8e947 check-qjson: Test noncharacters other than U+FFFE, U+FFFF in strings
Test cases cover the two noncharacters in the BMP.  Add tests for the
other 64 noncharacters.

Three existing test cases involve noncharacters U+FFFF and U+10FFFF.
Instead of deleting them as now duplicates, adjust them to use U+FFFC
and U+10FFFFD.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2013-04-13 19:40:17 +00:00