1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
|
[[logging]]
== libosmocore Logging System
In any reasonably complex software it is important to understand how to
enable and configure logging in order to get a better insight into what
is happening, and to be able to follow the course of action. We
therefore ask the reader to bear with us while we explain how the
logging subsystem works and how it is configured.
Most Osmocom Software (like `osmo-bts`, `osmo-bsc`, `osmo-nitb`,
`osmo-sgsn` and many others) uses the same common logging system.
This chapter describes the architecture and configuration of this common
logging system.
The logging system is composed of
* log targets (where to log),
* log categories (who is creating the log line),
* log levels (controlling the verbosity of logging), and
* log filters (filtering or suppressing certain messages).
All logging is done in human-readable ASCII-text. The logging system is
configured by means of VTY commands that can either be entered
interactively, or read from a configuration file at process start time.
[[log_categories]]
=== Log categories
Each sub-system of the program in question typically logs its messages as a
different category, allowing fine-grained control over which log
messages you will or will not see. For example, in OsmoBSC, there are
categories for the protocol layers `rsl`, `rr`, `mm`,
`cc` and many others. To get a a list of categories interactively
on the vty, type: `logging level ?`
[[log_levels]]
=== Log levels
For each of the log categories (see <<log_categories>>), you can set an independent log level,
controlling the level of verbosity. Log levels include:
fatal::
Fatal messages, causing abort and/or re-start of a process.
This __shouldn't happen__.
error::
An actual error has occurred, its cause should be further
investigated by the administrator.
notice::
A noticeable event has occurred, which is not
considered to be an error.
info::
Some information about normal/regular system
activity is provided.
debug::
Verbose information about internal processing of the system,
used for debugging purpose. This will log the most.
The log levels are inclusive, e.g. if you select __info__, then this
really means that all events with a level of at least __info__ will be
logged, i.e. including events of __notice__, __error__ and __fatal__.
So for example, in OsmoBSC, to set the log level of the Mobility
Management category to info, you can use the following command:
`log level mm info`.
[[log_filters]]
=== Log filters
The default behavior is to filter out everything, i.e. not to
log anything. The reason is quite simple: On a busy production setup,
logging all events for a given subsystem may very quickly be flooding
your console before you have a chance to set a more restrictive filter.
To request no filtering, i.e. see all messages, you may use:
`log filter all 1`
As another example, to only see messages relating to a particular
subscriber identified by his IMSI, you may use:
`log filter imsi 262020123456789`
=== Log targets
Each of the log targets represent certain destination for log messages.
It can be configured independently by selecting levels (see <<log_levels>>) for categories
(see <<log_categories>>) as well as filtering (see <<log_filters>>) and
other options like `logging timestamp` for example.
==== Logging to the VTY
Logging messages to the interactive command-line interface (VTY) is most
useful for occasional investigation by the system administrator.
Logging to the VTY is disabled by default, and needs to be enabled
explicitly for each such session. This means that multiple concurrent
VTY sessions each have their own logging configuration. Once you close
a VTY session, the log target will be destroyed and your log settings be
lost. If you re-connect to the VTY, you have to again activate and
configure logging, if you wish.
To create a logging target bound to a VTY, you have to use the following
command: `logging enable` This doesn't really activate the
generation of any output messages yet, it merely creates and attaches a
log target to the VTY session. The newly-created target still doesn't
have any filter installed, i.e. __all log messages will be suppressed
by default__
Next, you can configure the log levels for desired categories in your VTY session.
See <<log_categories>> for more details on categories and <<log_levels>> for the log level details.
For example, to set the log level of the Call Control category to debug, you
can use:
`log level cc debug`
Finally, after having configured the levels, you still need to set the
filter as it's described in <<log_filters>>.
TIP: If many messages are being logged to a VTY session, it may be hard
to impossible to still use the same session for any commands. We
therefore recommend to open a second VTY session in parallel, and use
one only for logging, while the other is used for interacting with the
system. Another option would be to use different log target.
==== Logging to the ring buffer
To avoid having separate VTY session just for logging output while still having immediate access to them,
one can use `alarms` target. It lets you store the log messages inside the ring buffer of a given size which
is available with `show alarms` command.
It's configured as follows:
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# log alarms 98
OsmoBSC(config-log)#
----
In the example above 98 is the desired size of the ring buffer (number of messages). Once it's filled,
the incoming log messages will push out the oldest messages available in the buffer.
==== Logging via gsmtap
When debugging complex issues it's handy to be able to reconstruct exact chain of events. This is enabled by using GSMTAP
log output where frames sent/received over the air are intersperced with the log lines. It also simplifies the bug handling
as users don't have to provide separate .pcap and .log files anymore - everything will be inside self-contained packet dump.
It's configured as follows:
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# log gsmtap 192.168.2.3
OsmoBSC(config-log)#
----
The hostname/ip argument is optional: if omitted the default 127.0.0.1 will be used. The log strings inside GSMTAP are already
supported by Wireshark. Capturing for `port 4729` on appropriate interface will reveal log messages including source file
name and line number as well as application. This makes it easy to consolidate logs from several different network components
alongside the air frames. You can also use Wireshark to quickly filter logs for a given subsystem, severity, file name etc.
[[fig-wireshark-gsmtap-log]]
.Wireshark with logs delivered over GSMTAP
image::../common/images/wireshark-gsmtap-log.png[]
Note: the logs are also duplicated to stderr when GSMTAP logging is configured.
==== Logging to a file
As opposed to Logging to the VTY, logging to files is persistent and
stored in the configuration file. As such, it is configured in
sub-nodes below the configuration node. There can be any number of log
files active, each of them having different settings regarding levels /
subsystems.
To configure a new log file, enter the following sequence of commands:
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# log file /path/to/my/file
OsmoBSC(config-log)#
----
This leaves you at the config-log prompt, from where you can set the
detailed configuration for this log file. The available commands at
this point are identical to configuring logging on the VTY, they include
`logging filter`, `logging level` as well as `logging color`
and `logging timestamp`.
TIP: Don't forget to use the `copy running-config startup-config` (or
its short-hand `write file`) command to make your logging configuration
persistent across application re-start.
NOTE: libosmocore currently does not provide file close-and-reopen
support by SIGHUP, as used by popular log file rotating solutions.
Please contact the Osmocom developers if you require this feature to be
implemented.
==== Logging to syslog
syslog is a standard for computer data logging maintained by the IETF.
Unix-like operating systems like GNU/Linux provide several syslog
compatible log daemons that receive log messages generated by
application programs.
libosmocore based applications can log messages to syslog by using the
syslog log target. You can configure syslog logging by issuing the
following commands on the VTY:
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# log syslog daemon
OsmoBSC(config-log)#
----
This leaves you at the config-log prompt, from where you can set the
detailed configuration for this log file. The available commands at
this point are identical to configuring logging on the VTY, they include
`logging filter`, `logging level` as well as `logging color`
and `logging timestamp`.
NOTE: Syslog daemons will normally automatically prefix every message
with a time-stamp, so you should disable the libosmocore time-stamping
by issuing the `logging timestamp 0` command.
==== Logging to stderr
If you're not running the respective application as a daemon in the
background, you can also use the stderr log target in order to log to
the standard error file descriptor of the process.
In order to configure logging to stderr, you can use the following
commands:
----
OsmoBSC> enable
OsmoBSC# configure terminal
OsmoBSC(config)# log stderr
OsmoBSC(config-log)#
----
|
