1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
|
[[common-counters]]
== Osmocom Counters
The following gives an overview of all the types of counters available:
=== Osmo Counters (deprecated)
Osmo counters are the oldest type of counters added to Osmocom projects.
They are not grouped.
* Printed as part of VTY show stats
* Increment, Decrement
* Accessible through the control interface: counter.<counter_name>
=== Rate Counters
Rate counters count rates of events.
* Printed as part of VTY show stats
* Intervals: per second, minute, hour, day or absolute value
* Increment only
* Accessible through the control interface
* Rate counters are grouped and different instances per group can exist
The control interface command to get a counter (group) is:
rate_ctr.per_{sec,min,hour,day,abs}.<group_name>.<idx>.[counter_name]
It is possible to get all counters in a group by omitting the counter name
=== Stat Item
Stat items are a grouped replacement for osmo counters.
* Printed as part of VTY show stats
* Replacement for osmo counters
* Not yet available through the control interface
* Grouped and indexed like rate counters
* Items have a unit
* Keeps a list of the last values measured, so could return an average, min,
max, std. deviation. So far this is not implemented in any of the reporting
options.
=== Statistic Levels
There are three levels on which a statistic can be aggregated in Osmocom
projects: globally, per-peer and per-subscriber.
==== Global
These are global statistics.
==== Peer
These statistics relate to a peer the program connects to such as the NSVC in
an SGSN.
This level also includes reporting global statistics.
==== Subscriber
These statistics are related to an individual mobile subscriber. An example
would be bytes transferred in an SGSN PDP context.
This level also includes global and peer-based statistics.
=== Stats Reporter
The stats reporter periodically collects osmo counter, rate counter and
stat item values and sends them to a backend. Currently implemented are
outputting to the configured log targets and a statsd connector.
==== Configuring a stats reporter
Periodically printing the statistics to the log can be done in the following
way:
.Log statistics
====
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# stats interval 60 <1>
OsmoBSC(config)# stats reporter log <2>
OsmoBSC(config-stats)# level global <3>
OsmoBSC(config-stats)# enable <4>
----
====
<1> The interval determines how often the statistics are reported.
<2> Write the statistic information to any configured log target.
<3> Report only `global` statistics (can be `global`, `peer`, or
`subscriber`).
<4> Enable the reporter, `disable` will disable it again.
The counter values can also be sent to any aggregation/visualization tool that
understands the statsd format, for example a statsd server with graphite or
prometheus using the statsd_exporter together with grafana.
The statsd format is specified in https://github.com/b/statsd_spec
.Report statistics to statsd
====
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# stats interval 10
OsmoBSC(config)# stats reporter statsd <1>
OsmoBSC(config-stats)# prefix BSC1 <2>
OsmoBSC(config-stats)# level subscriber <3>
OsmoBSC(config-stats)# remote-ip 1.2.3.4 <4>
OsmoBSC(config-stats)# remote-port 8125 <5>
OsmoBSC(config-stats)# enable
----
====
<1> Configure the statsd reporter.
<2> Prefix the reported statistics. This is useful to distinguish statistics
from multiple instances of the same service.
<3> Report only `global` statistics or include `peer` or `subscriber`
statistics as well.
<4> IP address of the statsd server.
<5> UDP port of the statsd server. Statsd by default listens to port 8125.
You can use Netdata (https://learn.netdata.cloud/) as a statsd server which does not require special configuration
to show rate counters.
By default all the rate counters will be exposed to the StatsD plugin (listening on 127.0.0.1:8125) and displayed on the
Netdata dashboard available via: http://localhost:19999
The list of available charts which includes all the rate counters reported via statsD is available through:
http://localhost:19999/api/v1/charts
=== Socket stats
libosmocore provides features to monitor the status of TCP connections. This
can be a helpful source of information when the links between network
components are unreliable (e.g. satellite link between BTS and BSC).
NOTE: This feature is only available for certain types of TCP connections. At
the moment only RSL/OML connections between OsmoBSC and the connected BTSs can
be monitored.
==== Configuration
The gathering of the TCP connection statistics is done via syscalls. This has
to be taken into account for the configuration. Since syscalls are rather
expensive and time consuming the overall performance of the application may
suffer when many TCP connections are present. This may be the case for BSCs
with a large number of BTSs connected to it.
The statistics are gathered in batches per interval. A batch size of 5 would
mean that only 5 TCP connections per interval are evaluated and the next 5
connections in the next interval and so on.
It is recommended to choose a large reporting interval and a reasonable small
batch size to distribute the syscall load as even as possible.
.Report statistics to statsd
====
----
OsmoBSC> enable
OsmoBSC# configure terminal
stats-tcp interval 10 <1>
stats-tcp batch-size 5 <2>
----
====
<1> Set the gathering interval (sec.)
<2> Set how many TCP sockets statistics to gather per interval.
==== Generated stats items
[options="header"]
|===
| Name | Description
| tcp:unacked | unacknowledged packets.
| tcp:lost | unacknowledged packets.
| tcp:retrans | lost packets.
| tcp:rtt | retransmitted packets.
| tcp:rcv_rtt | roundtrip-time (receive).
| tcp:notsent_bytes | bytes not yet sent.
| tcp:rwnd_limited | time (usec) limited by receive window.
| tcp:sndbuf_limited | Time (usec) limited by send buffer.
| tcp:reord_seen | reordering events seen.
|===
The item group index is the file descriptor number. The item group name
consists of a static prefix (e.g. "ipa-rsl"), followed by the IP addresses
and ports of both peers.
.VTY output of a stats item group of a TCP connection
====
----
stats tcp (15)('ipa-rsl,r=10.9.1.143:38455<->l=10.9.1.162:3003'):
unacknowledged packets: 0
lost packets: 0
retransmitted packets: 0
roundtrip-time: 583
roundtrip-time (receive): 0
bytes not yet sent: 0
time (usec) limited by receive window: 0
Time (usec) limited by send buffer: 0
reordering events seen: 0
----
====
|
