fix APER encoding of integer (backport from openairinterface)

The number of bytes used by an APER encoded integer depends on its
actually encoded value, not on the maximum value that could be possibly
encoded.

The old code would e.g. always use 24 bits if the maximum encoded value
would require 24 bits.

To give an example RANAP MaxBitrate (INTEER 1 .. 16000000) value 64000
was previously encoded as "80 00 f9 ff", while it is now the correct
representation "40 f9 ff".

Thanks to Dieter Spaar for detecting this problem in the Osmo-IUH
generated RANAP output, and thanks to openairinterface for fixing the
bug in their code (sadly not contributed to upstream asn1c, though).

1 files changed, 22 insertions, 5 deletions

diff --git a/src/INTEGER.c b/src/INTEGER.c
index 9ba14bc..eef89f5 100644
--- a/src/INTEGER.c
+++ b/src/INTEGER.c
@@ -995,18 +995,35 @@ INTEGER_encode_aper(asn_TYPE_descriptor_t *td,
 		} else {
 			/* TODO: extend to >64 bits */
 			int64_t v64 = v;
-			int i;
+			int i, j;
+			int max_range_bytes = (ct->range_bits >> 3) +
+					(((ct->range_bits % 8) > 0) ? 1 : 0);
+
+			for (i = 1; ; i++) {
+				int upper = 1 << i;
+				if (upper >= max_range_bytes)
+					break;
+			}
+
+			for (j = sizeof(int64_t) -1; j != 0; j--) {
+				uint8_t val;
+				val = v64 >> (j * 8);
+				if (val != 0)
+					break;
+			}
 
-			/* Putting length - 1 in the minimum number of bits ex: 5 = 3bits */
-			if (per_put_few_bits(po, st->size - 1, (ct->range_bits >> 3)-1))
+			/* Putting length in the minimum number of bits ex: 5 = 3bits */
+			if (per_put_few_bits(po, j, i))
 				_ASN_ENCODE_FAILED;
 
 			// Consume the bits to align on octet
 			if (aper_put_align(po) < 0)
 				_ASN_ENCODE_FAILED;
+
 			/* Put the value */
-			for (i = 0; i < st->size; i++) {
-				if(per_put_few_bits(po, (v64 >> (8 * (st->size - i - 1))) & 0xff, 8)) _ASN_ENCODE_FAILED;
+			for (i = 0; i <= j; i++) {
+				if(per_put_few_bits(po, (v64 >> (8 * (j - i))) & 0xff, 8))
+					_ASN_ENCODE_FAILED;
 			}
 		}
 		_ASN_ENCODED_OK(er);