util/cutils: Fix incorrect integer->float conversion caught by clang

Clang does not like do_strtosz()'s code to guard against overflow:

    qemu/util/cutils.c:245:23: error: implicit conversion from 'unsigned long' to 'double' changes value from 18446744073709550592 to 18446744073709551616 [-Werror,-Wimplicit-int-float-conversion]

The warning will be enabled by default in clang 10. It is not
available for clang <= 9.

val * mul >= 0xfffffffffffffc00 is indeed wrong.  0xfffffffffffffc00
is not representable exactly as double.  It's half-way between the
representable values 0xfffffffffffff800 and 0x10000000000000000.
Which one we get is implementation-defined.  Bad.

We want val * mul > (the largest uint64_t exactly representable as
double).  That's 0xfffffffffffff800.  Write it as nextafter(0x1p64, 0)
with a suitable comment.

Signed-off-by: Fangrui Song <i@maskray.me>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Patch split, commit message improved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191122080039.12771-3-armbru@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
This commit is contained in:
Fangrui Song 2019-11-22 09:00:39 +01:00 committed by Markus Armbruster
parent 2061735ff0
commit 25f74087c6

View File

@ -239,10 +239,12 @@ static int do_strtosz(const char *nptr, const char **end,
goto out;
}
/*
* Values >= 0xfffffffffffffc00 overflow uint64_t after their trip
* through double (53 bits of precision).
* Values near UINT64_MAX overflow to 2**64 when converting to double
* precision. Compare against the maximum representable double precision
* value below 2**64, computed as "the next value after 2**64 (0x1p64) in
* the direction of 0".
*/
if ((val * mul >= 0xfffffffffffffc00) || val < 0) {
if ((val * mul > nextafter(0x1p64, 0)) || val < 0) {
retval = -ERANGE;
goto out;
}