Age | Commit message (Collapse) | Author | Files | Lines |
|
This way we account for the number of dropped log messages at
write_queue overflow. However, log targets have no numerical
ID and hence we're not able to extract them in a reasonable manner.
Related: OS#4311
Change-Id: I89b696311b823267e05d6a3e85b92c1784b220ed
|
|
So far, we used blocking, buffered fwrite() to write to stderr
and file targets. This causes problems if there are [slow] consumers
causing delays, such as gnome-terminal (when the program is started
interactively) or systemd/journald (where we observe 64..128ms blocks on
stderr).
This patch introduces stderr/file based logging via write_queue
and osmo_select_main(), i.e. switch from glibc-buffered, blocking
to internally buffered, non-blocking writes.
* when osmo_stderr_target is created via application.c, we create it
in blocking stream mode for backwards compatibility, particularly
for [smaller] programs that don't use osmo_select_main()
* when the VTY code encounters 'log stderr' or 'log file FILENAME',
we switch that respective target to non-blocking write-queue mode,
as this means the application is in fact using osmo_select_main()
* The config file can now state 'log stderr blocking-io' or
'log file FILENAME blocking-io' to explicitly enforce using blocking
stream based I/O
* The application can at any time use API functions to switch either way
Closes: OS#4311
Change-Id: Ia58fd78535c41b3da3aeb7733aadc785ace610da
|
|
This adds an inter-thread queue "it_q" to libosmocore. With it_q,
one can perform thread-safe enqueing of messages to another thread,
who will receive the related messages triggered via an eventfd
handled in the usual libosmocore select loop abstraction.
Change-Id: Ie7d0c5fec715a2a577fae014b0b8a0e9c38418ef
|
|
In 'struct gsmtap_hdr' field 'snr_db' is defined as a signed integer,
however all functions that fill this structure accept an unsigned
integer. This is wrong, because SNR can be negative.
Let's use 'int8_t' instead of 'uint8_t'. Changing from unsigned
to signed should be relatively safe compared to the opposite.
Most of the callers I am aware of always do pass 0 anyway.
Change-Id: I9f432be5c346d563bf518111c14ff04d4a63f592
Related: SYS#5073
|
|
This function is called automatically on the main thread, but needs to
ba called explicitly in order to run the select loop on another thread.
Make it available for applications through talloc.h
Change-Id: Ie710ca9ad01d3fadb9f4ff344a55d6c01004727b
|
|
This is a helper function to broadcast an event to all of the
siblings of a specified FSM instance.
Change-Id: I2ce398741a8672d7b7c4058d056f46e2fe7353c1
|
|
Historically, BSSGP uses a non-constant, user-configurable integer
varieable for the logging sub-system. Let's replace this with a
statically-allocated library logging constant.
This is required if we want to use the subsystem number in e.g.
static initialized for osmo_fsm.log_subsys.
Change-Id: I506190aae9217c0956e4b5764d1a0c0772268e93
|
|
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h:10:0: error: "__always_inline" redefined [-Werror]
#define __always_inline inline __attribute__((always_inline))
/usr/include/x86_64-linux-gnu/sys/cdefs.h:311:0: note: this is the location of the previous definition
# define __always_inline __inline __attribute__ ((__always_inline__))
Change-Id: I738d2a72f835a29e30b8ba20456e5c4c9aa844c9
|
|
Change-Id: Ieb872551bdbe514f2c77f9aeb2b9ee42f6573909
|
|
When importing the hashtable code in I8ef73a62fe9846ce45058eb21cf999dd3eed5741
I didn't import actual implementations of the fls() and fls64()
implementations, as at least gcc-10 was smart enough to detect
we only use it on constant types and hence the computation can happen
at build time via const_ilog2()
However, in our jenkins build verification' this doesn't appear to
happen, as we get below errors:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h: In function ‘__ilog2_u32’:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h:20:9: error: implicit declaration of function ‘fls’ [-Werror=implicit-function-declaration]
return fls(n) - 1;
^~~
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h: In function ‘__ilog2_u64’:
/build/deps/install/stow/libosmocore/include/osmocom/core/log2.h:28:9: error: implicit declaration of function ‘fls64’ [-Werror=implicit-function-declaration]
return fls64(n) - 1;
^~~~~
Let's provide some generic implementations for this case. If needed
one could also introduce architecture-specific assembly implementations
like in the Linux kernel, but so far we managed to keep libosmocore free
of any assembly tweaks.
Change-Id: Ifa4898eb66c8d949618edd47961b7a0330ed35b5
|
|
/usr/local/include/osmocom/core/linuxlist.h:479:12: error: invalid conversion from ‘void*’ to ‘hlist_node*’ [-fpermissive]
479 | n->next = LLIST_POISON1;
Fixes: I8ef73a62fe9846ce45058eb21cf999dd3eed5741
Change-Id: I75b0a5fe097562007c53987d8d41811e9f35798d
|
|
'new' is a reserved keyword in C++, so including this header from
a C++ project (like osmo-pcu) breaks compilation. Let's rename
it in the same way as it's already done in this file: add '_'.
Change-Id: I7f7d9143edca75ce932601386a8766b0a62c0e24
Fixes: I8ef73a62fe9846ce45058eb21cf999dd3eed5741
|
|
For more than a decade we've used the linuxlist.h for double-linked
lists. Let's also add the hlist (double-linked lists with single
pointer sized head, and the hashtable that builds on top of it.
This reflects the versions included in Linux 5.8 with some modifications
to make them build in userspace (remove RCU versions, adjust for
userspace include files and types, convert to doxygen).
Change-Id: I8ef73a62fe9846ce45058eb21cf999dd3eed5741
|
|
NSVC filtering was only implemented on sending messages, this also adds
log_set_context() calls to ns2_recv_vc()
Filtering by NSE is implemented similar to NSVC.
Change-Id: I63c0e85f82f5d08c5a6f535da94b8648498439d2
Related: SYS#5232
|
|
This adds an easy way to listen to netlink events form the Linux kernel
from within libosmocore applications.
The new dependency can be disabled via the "--disable-lbimnl" configure flag.
Change-Id: I4f787ee68f0d6d04f0a5655eb57d55b3b326a42f
|
|
Change-Id: I1ee1278b029e42321932b87f94aa3e0eeed4108a
Related: SYS#5232
|
|
This allows usual integer parsing at app level and calling this function
to make sure correct values will be passed to
osmo_serial_set_baudrate().
Change-Id: I41415c99d26128b33a8bf5ef7b38948bd1fe5d50
|
|
Some applications may need submillisecond timers, such as those
interacting with modbus serial lines (RS-485, RTU), which require
timers of values around 1.5 char-time (T1.5), where a data char is
composed of 11 bits sent on the line: 1 start bit, 8 data bits,
1 stop bit, and and parity bit (or 2nd stop bits if no parity).
For instance, for a baudrate of 9600:
1.5 * 11 / 9600 = 1.718 ms = 1718 us
So having a granularity of MS is not enough here.
Change-Id: I71848d7c1ee0649929ce07680ee7320bb2a42f0e
|
|
Change-Id: Ic04caab15abbd0c0d3a01f6e128935a3ceed903e
|
|
select is an ancient interface with weird restrictions, such as
the fact that it cannot be used for file descriptor values > 1024.
This may have been sufficient 40 years ago, but certainly is not in
2020. I wanted to migrate to epoll(), but unfortunately it doesn't
work well with the fact that existing programs simply set osmo_fd.flags
without making any API calls at the time they change those flags.
So let's do the migration to poll() as a first step, and then consider
epoll() as a second step further down the road, after introducing new
APIs and porting applications over.
The poll() code introduced in this patch is not extremely efficient,
as it needs to do extensive linked list iterations after poll() returns
in order to find the osmo_fd from the fd. Optimization is possible,
but let's postpone that to a follow-up patch.
At compile time, a new --enable-force-io-select argument can be given
to configure, forcing the use of the old select() backend instead of the
new poll() based backend.
Change-Id: I9e80da68a144b36926066610d0d3df06abe09bca
|
|
This change implements 'systemd-journal' logging target, that is
similar to the existing 'syslog' target. The key difference is
that 'systemd-journal' allows us to offload rendering of the meta
information, such as location (file name, line number), subsystem,
and logging level, to systemd. Moreover, we can attach arbitrary,
user-specific fields [1] to the logging messages, so they can be
used for advanced log filtering (e.g. by IMSI/TMSI/TLLI):
$ journalctl OSMO_SUBSYS=DMSC -f
Since we don't want to make libsystemd a required dependency, this
feature is optional, and needs to be enabled at build-time:
$ ./configure --enable-systemd-logging
The new logging target can be configured in the same way as any
other one - via the VTY interface, or using the configuration file:
log systemd-journal [raw]
logging level set-all notice
logging filter all 1
Two logging handlers are available: generic and raw. The first one
behaves similarly to both 'syslog' and 'stderr', i.e. all the meta
information is rendered by libosmocore itself, and then passed to
systemd together with the logging message. The later is more like
the 'gsmtap' target, so all available meta information is handed
over to systemd in form of fields [1]:
- CODE_FILE / CODE_LINE - location info,
- PRIORITY - syslog-compatible logging level,
- OSMO_SUBSYS - Osmocom-specific sub-system (e.g. DMSC),
- OSMO_SUBSYS_HEX - same as OSMO_SUBSYS, but encoded in hex,
- MESSAGE - the logging message itself,
and then can be rendered in any supported format (e.g. JSON).
More details about the API can be found in [2].
[1] https://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html
[2] https://www.freedesktop.org/software/systemd/man/sd-journal.html
Change-Id: I609f5cf438e6ad9038d8fc95f00add6aac29fb23
|
|
If we watn to migrate to something like epoll(), user application
code must call a function of the libosmocore API whenever it changes
its read/write interest in a file descriptor.
Let's introduce API so applications can be ported to this API,
before making direct 'ofd->when' manipulations illegal as a second step.
Change-Id: Idb89ba7bc7c129a6304a76900d17f47daf54d17d
|
|
Add helper to format osmo_sockaddr into a string.
Change-Id: I917f25ebd1239eae5855d973ced15b93731e33a0
|
|
Change-Id: Ibfdfdd40c52709b32ac934974cc78ee821fa83ba
|
|
This will be useful to handle latitude and longitude numbers for GAD, which is
the location estimate representation used for LCS (Location Services).
The OsmoSMLC VTY user interface will provide floating-point strings like
"23.456" while GAD stores them as micro-degress 23456000. The osmo_gad_to_str*
will also convert latitude and longitude to floating-point string.
There was code review concerns against adding this API, upon which I tried to
use floating point string formats. But I encountered various problems with
accuracy and trailing zeros. For global positioning data (latitude and
longitude), even inaccuracy on the sixth significant decimal digit causes
noticeable positional shift. To achieve sufficient accuracy on the least
significant end, I need to use double instead of float. To remove trailing
zeros, the idea was to use '%.6g' format, but that can cause rounding. '%.6f'
on a double looks ok, but always includes trailing zeros. A test program shows:
%.6g of ((double)(int32_t)23230100)/1e6 = "23.2301" <-- good
%.6g of ((double)(int32_t)42419993)/1e6 = "42.42" <-- bad rounding
%.6g of ((double)(int32_t)23230199)/1e6 = "23.2302" <-- bad rounding
%.6f of ((double)(int32_t)23230100)/1e6 = "23.230100" <-- trailing zeros
%.6f of ((double)(int32_t)42419993)/1e6 = "42.419993" <-- good
%.6f of ((double)(int32_t)23230199)/1e6 = "23.230199" <-- good
It looks like when accepting that there will be trailing zeros, using double
with '%.6f' would work out, but in the end I am not certain enough that there
aren't more hidden rounding / precision glitches. Hence I decided to reinforce
the need to add this API: it is glitch free in sufficient precision for
latitude and longitude data, because it is based on integer arithmetic.
The need for this precision is particular to the (new) OsmoSMLC vty
configuration, where reading and writing back user config must not modify the
values the user entered. Considering to add these functions to osmo-smlc.git,
we might as well add them here to libosmocore utils, and also use them in
osmo_gad_to_str_*() functions.
Change-Id: Ib9aee749cd331712a4dcdadfb6a2dfa4c26da957
|
|
Same as osmo_wqueue_enqueue() but without logging queue overruns.
Change-Id: Ic082eb39795b08631284eeb421fef3c28f2e90dc
|
|
So far there is only osmo_use_count_name_buf(). Also provide a use count to
string using a talloc context, allowing to use OTC_SELECT.
- instead of foo_name(), rather use foo_to_str().
- osmo_use_count_name_buf() returns the buf and not the chars_needed. So add
osmo_use_count_to_str_buf() with a signature that is usable by
OSMO_NAME_C_IMPL().
- provide osmo_use_count_to_str_c() using OSMO_NAME_C_IMPL().
Change-Id: I1d2e7ee979f8c316ef99f7c65675b36d092ddfca
|
|
Reimplement NS with FSM.
Change-Id: I3525beef205588dfab9d3880a34115f1a2676e48
|
|
As shown in the recently added bitgen_test.c, using osmo_loadXXbe_ext() with a
smaller n produces results aligned on the most significant bytes, which is
cumbersome, since it does not return a previously stored value. This problem
exists only for the big-endian functions, the little-endian osmo_loadXXle_ext()
properly return values adjusted on the least significant octets.
Add osmo_loadXXbe_ext_2() variants that properly right-adjust the returned
value. Prominently highlight this behavior in API doc. Test the new functions
in bitgen_test.c.
For example, this eases handling of 24bit integers (e.g. loaded from buffer to
uint32_t, and stored into buffer from uint32_t). Also explicitly show this 24
bit case in bitgen_test.c
Change-Id: I2806df6f0f7bf1ad705d52fa386d4525b892b928
|
|
Add OSMO_ASSERT()s to ensure bounds checking.
For example, for osmo_store32le_ext(), passing n > 5 would read past the end of
the uint32_t. Similarly, osmo_load32le_ext() for n > 4 would write past the
uint32_t's end.
Change-Id: I2dc21582cd8a679b6624cefbc0c1678b093a3d08
|
|
Similiar to osmo_sock_local_ip but for osmo_sockaddr.
Change-Id: I9cd2c5ceb28183e2fd2d28f9c9088c3fcac643d2
|
|
Compare two osmo_sockaddr.
Change-Id: I2d12ebae2710ffd17cf071e6ada0804e73f87dd6
|
|
osmo_sock_init_osa() takes osmo_sockaddr* as local and remote endpoints
to setup a socket.
Change-Id: I1eece543e3241ef0e095eb63bb831f7c15a16794
|
|
In case the port isn't known at the time osmo_sockaddr_str_from_str2()
parse only the ip and don't touch the port.
This is the case when a user has different vty commands for ip and port.
Change-Id: Ifd4e282586b8bd40b912a9f1c25f9e8208420106
|
|
These are APIs useful to inline in log calls.
Change-Id: Ie07a38b05b7888885dba4ae795e9f3d9a561543d
|
|
Change-Id: Ie22eac9a881fe0822f8abc9de73620b93b1f2b37
|
|
When dealing with IPv4 and IPv6 address, the struct sockaddr
allows to hold IPv4 and IPv6.
However even when struct sockaddr is being used, a cast to the
IPv4 or IPv6 family must happen. To work around this additional code,
use a union for the most common types.
Change-Id: If80172373735193401af872b18e1ff00c93880e7
|
|
Change-Id: I80fc0ea8865ec4efdcd4c982e69d863275fd9919
Related: SYS#4877
|
|
Change-Id: If2f806d044cd0fb6929dac44ef8f8a15941ffe9b
Related: SYS#4877
|
|
Change-Id: Ib28ea88c8e8e9b33d70d58156d03af9a41e9e012
|
|
Implement better API around 3GPP TS 24.008 Mobile Identity coding.
struct osmo_mobile_identity is a decoded representation of the raw Mobile
Identity, with a string representation as well as dedicated raw uint32_t TMSI.
The aim is to remove all uncertainty about decoded buffer sizes / data types.
I have patches ready for current osmo CNI programs, replacing the Mobile
Identity coding with this new API. Deprecate the old MI API.
osmo-bsc: I71c3b4c65dbfdfa51409e09d4868aea83225338a
osmo-msc: Ic3f969e739654c1e8c387aedeeba5cce07fe2307
osmo-sgsn: I4cacb10bac419633ca0c14f244f9903f7f517b49
Note that some GPRS and SGs related coding is done here in libosmocore and
hence currently remains using the old implementation (see previous version of
this patch: Ic3f969e739654c1e8c387aedeeba5cce07fe2307).
New API functions provide properly size-checking implementations of:
- decoding a raw MI from a bunch of MI octets;
- locating and decoding MI from a full 3GPP TS 24.008 Complete Layer 3 msgb;
- encoding to a buffer;
- encoding to the end of a msgb.
Other than the old gsm48_generate_mid(), omit a TLV tag and length from
encoding. Many callers manually stripped the tag and value after calling
gsm48_generate_mid(). The aim is to leave writing a TL to the caller entirely,
especially since some callers need to use a TvL, i.e. support a variable-size
length of 8 or 16 bit.
New validity checks so far not implemented anywhere else:
- stricter validation of number of digits of IMSI, IMEI, IMEI-SV MI.
- stricter on filler nibbles to be 0xf.
As a result, applications using osmo_mobile_identity will be stricter in
rejecting coding mistakes (some of which we currently have in our test suites,
and which we'll need to fix).
Rationale:
While implementing osmo-bsc's MSC pooling feature in osmo-bsc, this API will be
used to reduce the number of times a Mobile Identity is extracted from a raw
RSL message.
Extracting the Mobile Identity from messages has numerous duplicate
implementations across our code with various levels of specialization.
https://xkcd.com/927/
To name a few:
- libosmocore: gsm48_mi_to_string(), osmo_mi_name_buf()
- osmo-bsc: extract_sub()
- osmo-msc: mm_rx_loc_upd_req(), cm_serv_reuse_conn(), gsm48_rx_mm_serv_req(),
vlr_proc_acc_req()
We have existing functions to produce a human readable string from a Mobile
Identity, more or less awkward:
- gsm48_mi_to_string() decodes a TMSI as a decimal number. These days we use
hexadecimal TMSI everywhere.
- osmo_mi_name_buf() decodes the BCD digits from a raw MI every time, so we'd
need to pass around the raw message bytes. Also, osmo_mi_name_buf() has the
wrong signature, it should return a length like snprintf().
- osmo-bsc's extract_sub() first uses gsm48_mi_to_string() which encodes the
raw uint32_t TMSI to a string, and then calls strtoul() via
tmsi_from_string() to code those back to a raw uint32_t.
Each of the above implementations employ their own size overflow checks, each
invoke osmo_bcd2str() and implement their own TMSI osmo_load32be() handling.
Too much code dup, let's hope that each and every one is correct.
In osmo-bsc, I am now implementing MSC pooling, and need to extract NRI bits
from a TMSI Mobile Identity. Since none of the above functions are general
enough to be re-used, I found myself again copy-pasting Mobile Identity code:
locating the MI in a 24.008 message with proper size checks, decoding MI
octets.
This time I would like it to become a generally re-usable API.
This patch was first merged as Ic3f969e739654c1e8c387aedeeba5cce07fe2307 and
caused test fallout, because it re-implemented old API with the new stricter
decoding. In this patch version, old API remains 1:1 unchanged to avoid such
fallout. Applications will soon switch to the new osmo_mobile_identity API and
become stricter on MI coding when that happens, not implicitly by a new
libosmocore version.
Change-Id: If4f7be606e54cfa1c59084cf169785b1cbda5cf5
|
|
This reverts commit d1ceca9d48eb3d8b212f386a1ebb35d8fc612297, as it
introduces regressions in both osmo-msc and osmo-nitb which have been
causing failing builds for several days now.
Change-Id: I4bd958d0cd2ab4b0c4725e6d114f4404d725fcf7
|
|
Implement better API around 3GPP TS 24.008 Mobile Identity coding.
struct osmo_mobile_identity is a decoded representation of the raw Mobile
Identity, with a string representation as well as dedicated raw uint32_t TMSI.
The aim is to remove all uncertainty about decoded buffer sizes / data types.
I have patches ready for all osmo programs, completely replacing the Mobile
Identity coding with this new API. Hence deprecate the old MI API.
New API functions provide properly size-checking implementations of:
- decoding a raw MI from a bunch of MI octets;
- locating and decoding MI from a full 3GPP TS 24.008 Complete Layer 3 msgb;
- encoding to a buffer;
- encoding to the end of a msgb.
Other than the old gsm48_generate_mid(), omit a TLV tag and length from
encoding. Many callers manually stripped the tag and value after calling
gsm48_generate_mid(). The aim is to leave writing a TL to the caller entirely,
especially since some callers need to use a TvL, i.e. support a variable-size
length of 8 or 16 bit.
New validity checks so far not implemented anywhere else:
- stricter validation of number of digits of IMSI, IMEI, IMEI-SV MI.
- stricter on filler nibbles to be 0xf.
Rationale:
While implementing osmo-bsc's MSC pooling feature in osmo-bsc, this API will be
used to reduce the number of times a Mobile Identity is extracted from a raw
RSL message.
Extracting the Mobile Identity from messages has numerous duplicate
implementations across our code with various levels of specialization.
https://xkcd.com/927/
To name a few:
- libosmocore: gsm48_mi_to_string(), osmo_mi_name_buf()
- osmo-bsc: extract_sub()
- osmo-msc: mm_rx_loc_upd_req(), cm_serv_reuse_conn(), gsm48_rx_mm_serv_req(),
vlr_proc_acc_req()
We have existing functions to produce a human readable string from a Mobile
Identity, more or less awkward:
- gsm48_mi_to_string() decodes a TMSI as a decimal number. These days we use
hexadecimal TMSI everywhere.
- osmo_mi_name_buf() decodes the BCD digits from a raw MI every time, so we'd
need to pass around the raw message bytes. Also, osmo_mi_name_buf() has the
wrong signature, it should return a length like snprintf().
- osmo-bsc's extract_sub() first uses gsm48_mi_to_string() which encodes the
raw uint32_t TMSI to a string, and then calls strtoul() via
tmsi_from_string() to code those back to a raw uint32_t.
Each of the above implementations employ their own size overflow checks, each
invoke osmo_bcd2str() and implement their own TMSI osmo_load32be() handling.
Too much code dup, let's hope that each and every one is correct.
In osmo-bsc, I am now implementing MSC pooling, and need to extract NRI bits
from a TMSI Mobile Identity. Since none of the above functions are general
enough to be re-used, I found myself again copy-pasting Mobile Identity code:
locating the MI in a 24.008 message with proper size checks, decoding MI
octets.
This time I would like it to become a generally re-usable API.
Change-Id: Ic3f969e739654c1e8c387aedeeba5cce07fe2307
|
|
Change-Id: Ibe722d38d28e5590a35e856dd15c2538e6ee0a3e
|
|
Reliable monitoring requires regular flush of all stat values, even
if they have not changed. Otherwise (1) the monitoring app has to
maintain state and (2) can go out of sync if it's restarted while
the app is still running.
Change-Id: I04f1e7bdf0d6f20e4f15571e94191de61c47ddad
|
|
The signalfd(2) mechanism of Linux allows signals to be delivered
and processed via normal file descriptor I/O. This avoids any of the
usual problems about re-entrancy of signal processing, as signals can
be processed from the osmocom select() loop abstraction just like any
other event.
Change-Id: If8d89dd1f6989e1cd9b9367fad954d65f91ada30
|
|
For a process running as root, it may be desirable to drop privileges
down to a normal user before executing an external command. Let's
add a new API function for that.
Change-Id: If1431f930f72a8d6c1d102426874a11b7a2debd9
|
|
Change-Id: Ib52d22710020b56965aefcef09bde8247ace4a9c
Related: OS#2966
|
|
Change-Id: Id2d016939c3a6185cc3cfa8631da0c8d187a8c5a
|
|
* What we used to call TCH/F and TCH/H in gsmtap are actually only
FACCH/F and FACCH/H, i.e. the signaling part of Bm/Lm channels
* Give them proper names with backwards compatibility #define
* Split VOICE into VOICE_F and VOICE_H. If we don't differentiate this,
a receiver is not able to determine the RSL channel ID of a frame
without looking at external state/context. That in turn has been a
design feature of GSMTAP Um format so far, and programs like
osmo-bts-virtual rely on it.
Change-Id: I952044a17334f35712e087dc41781805000aebc1
Related: OS#2557
|