mc-nmf: Fix undefined shift

It's possible to have a pathological case of a series of 0x80 bytes,
with only the extension bit set. That adds nothing to the size, so
we must check the number of octets. (It would be simpler to just check
that it fits in five octets, rather than check the size as well, but
that's the way the spec is written, even though 32 isn't evenly divisible
by 7.)

The literal 0x7F is an int, which means that the result of bitwise-and
with a uint8_t is an int, according to the C integer promotion rules.
An int is not necessarily 64 bits, which means that, in the limit case
before breaking out of the loop, shifting seven bits left by 28 (= 7 * 4)
can end up truncating. Make sure we have a 64 bit integer before the
shift operation.

Note that we could just allow the truncation to happen (after at least
assuring that our literal is an unsigned int, so that left-shift is
well defined in the case where the full 32 bits is needed for the size),
but we probably want to error out rather than ignoring the left most three
bits of the fifth octet.

Fix #18994

0 files changed, 0 insertions, 0 deletions