lib: check for integer-overflow in nlmsg_reserve()

In general, libnl functions are not robust against calling with
invalid arguments. Thus, never call libnl functions with invalid
arguments. In case of nlmsg_reserve() this means never provide
a @len argument that causes overflow.

Still, add an additional safeguard to avoid exploiting such bugs.

Assume that @pad is a trusted, small integer.
Assume that n->nm_size is a valid number of allocated bytes (and thus
much smaller then SIZE_T_MAX).
Assume, that @len may be set to an untrusted value. Then the patch
avoids an integer overflow resulting in reserving too few bytes.

lib/msg.c

@@ -411,6 +411,9 @@ void *nlmsg_reserve(struct nl_msg *n, size_t len, int pad)
 	size_t nlmsg_len = n->nm_nlh->nlmsg_len;
 	size_t tlen;
 
+	if (len > n->nm_size)
+		return NULL;
+
 	tlen = pad ? ((len + (pad - 1)) & ~(pad - 1)) : len;
 
 	if ((tlen + nlmsg_len) > n->nm_size)