| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
|
|
Each VLR requesting auth tuples should use a distinct IND pool for 3G auth. So
far we tied the IND to the GSUP peer connection; MSC and SGSN were always
distinct GSUP peers, they ended up using distinct INDs.
However, we have implemented a GSUP proxy, so that, in a distributed setup, a
remotely roaming subscriber has only one direct GSUP peer proxying for both
remote MSC and SGSN. That means as soon as a subscriber roams to a different
site, we would use the GSUP proxy name to determine the IND instead of the
separate MSC and SGSN. The site's MSC and SGSN appear as the same client, get
the same IND bucket, waste SQNs rapidly and cause auth tuple generation load.
So instead of using the local client as IND, persistently keep a list of VLR
names and assign a different IND to each. Use the gsup_req->source_name as
indicator, which reflects the actual remote VLR's name (remote MSC or SGSN).
Persist the site <-> IND assignments in the database.
Add an IND test to db_test.c
There was an earlier patch version that separated the IND pools by cn_domain,
but it turned out to add complex semantics, while only solving one aspect of
the "adjacent VLR" problem. We need a solution not only for CS vs PS, but also
for 2,3G vs 4G, and for sites that are physically adjacent to each other. This
patch version does not offer any automatic solution for that -- as soon as more
than 2^IND_bitlen (usually 32) VLRs show up, it is the responsibility of the
admin to ensure the 'ind' table in the hlr.db does not have unfortunate IND
assignments. So far no VTY commands exist for that, they may be added in the
future.
Related: OS#4319
Change-Id: I6f0a6bbef3a27507605c3b4a0e1a89bdfd468374
|
|
To determine distinct IND pools for each connected VLR, we need to pick ever
increasing values for any new peer showing up. Each subscriber's individual
IND_bitlen is then required to modulo the least significant N of bits that fit
in its IND_bitlen to effectively round-robin in the available IND pool space.
So far we did that but issued a warning message. This is actually exactly what
we want and it doesn't need to be treated like it weren't so.
Change-Id: I716d8a8a249235c8093d7a6a78b3535d893d867e
|
|
Change-Id: I0fe34e0f065160ef959b2b7b4dd040f3f2985f43
|
|
So far, the time string format comes from ctime_r, and we manually add "UTC" to it.
The ctime_r format is wildly chaotic IMHO, mixing weekday, day-of-month and
hour and year in very unsorted ways.
Adding "UTC" to it is non-standard.
Instead use an ISO-8601 standardized time string via strftime().
Change-Id: I6731968f05050399f4dd43b241290186e0c59e1a
|
|
Checking for existence of a subscriber and seeing that there is none is not
inherently an error. However, osmo-hlr currently logs on all occasions:
DAUC ERROR Cannot read subscriber from db: MSISDN='1001': No such subscriber
This spams the ERROR log level. Particularly when a D-GSM setup does subscriber
existence checks for every incoming mslookup request, that potentially creates
constant ERROR logging.
The "No such subscriber" part comes from db_sel(), which might also return an
sqlite3_errmsg(). We still want those sqlite3_errmsg()es in the ERROR log.
Hence print an ERROR log only if db_sel() returns an rc != -ENOENT.
Change-Id: I5044e9b4519b948edc4e451cef0f7830d315619b
|
|
Change-Id: I392b5523870c2ef3267179160028d26f3f761b77
|
|
Extend the "last LU seen on ..." line with the amount of seconds that
passed since now, or "(invalid timestamp)".
Patch split from Id7fc50567211a0870ac0524f6dee94d4513781ba, because it
depends on timestamp_age which was just added in
Ife4a61d71926d08f310a1aeed9d9f1974f64178b.
Change-Id: I24f9e86c1aa0b1576290094e024562f41b988f37
|
|
If a peer attempts to add a route to an ipa-name that we already have in the
routing system, don't send the routing error to the peer that already has the
name, but to the peer that attempts to re-use it and would cause the collision.
This is fixing a situation where for example a locally attached MSC has name
'MSC-1', and a remote site is proxying GSUP here for a remote MSC that also has
the name 'MSC-1'. Send the routing error back to the proxy, not local 'MSC-1'.
Change-Id: Icafaedc11b5925149d338bdcb987ae985a7323d6
|
|
Change-Id: I43ad27f6d768df02abb3459ac4c43bb80cc1cbeb
|
|
Add mslookup client to find remote home HLRs of unknown IMSIs, and
proxy/forward GSUP for those to the right remote HLR instances.
Add remote_hlr.c to manage one GSUP client per remote HLR GSUP address.
Add proxy.c to keep state about remotely handled IMSIs (remote GSUP address,
MSISDN, and probably more in future patches). The mslookup_server that
determines whether a given MSISDN is attached locally now also needs to look in
the proxy record: it is always the osmo-hlr immediately peering for the MSC
that should respond to mslookup service address queries like SIP and SMPP.
(Only gsup.hlr service is always answered by the home HLR.)
Add dgsm.c to set up an mdns mslookup client, ask for IMSI homes, and to decide
which GSUP is handled locally and which needs to go to a remote HLR.
Add full VTY config and VTY tests.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: I2fe453553c90e6ee527ed13a13089900efd488aa
|
|
Implement the mslookup server's mDNS responder, to actually service remote
mslookup requests:
- VTY mslookup/server config with service names,
- the mslookup_mdns_server listening for mslookup requests,
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: I5cae6459090588b4dd292be90a5e8903432669d2
|
|
Implement the mslookup server to service remote mslookup requests.
This patch merely adds the logic to answer incoming mslookup requests, an
actual method to receive requests (mDNS) follows in a subsequent patch.
- API to configure service names and addresses for the local site (per MSC).
- determine whether a subscriber is on a local MSC
(checking the local proxy will be added in subsequent patch that adds proxy
capability).
- VTY config follows in a subsequent patch.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: Ife4a61d71926d08f310a1aeed9d9f1974f64178b
|
|
This stuff is not testing osmo-hlr specific nodes, remove.
Change-Id: Ia11a209778b78ab02424e2abf3f9004fe97cf570
|
|
Make room for (more) arbitrary IPA headers, like longer IPA names as configured
by the user.
Change-Id: I7d86f2dadcae29fe1550ea2c9773394ab31a837b
|
|
D-GSM will store in the HLR DB whether a locally connected MSC has attached the
subscriber (last_lu_seen[_ps]), or whether the attach happened via a GSUP proxy
from a different site.
Add columns for this separately in this patch.
Change-Id: I98c7b3870559ede84adf56e4bf111f53c7487745
|
|
For the GSUP clients in upcoming D-GSM enabled osmo-hlr, it will be necessary
to trigger an event as soon as a GSUP client connection becomes ready for
communication. Add the osmo_gsup_client->up_down_cb.
Add osmo_gsup_client_create3() pass the up_down_cb in the arguments. Also add
a cb data argument.
We need the callbacks and data pointer in the osmo_gsup_client_create()
function right before startup, because this function immediately starts up the
connection. Who knows whether callbacks might trigger right away.
Because there are so many arguments, and to prevent the need for ever new
versions of this function, pass the arguments as an extendable struct.
Change-Id: I6f181e42b678465bc9945f192559dc57d2083c6d
|
|
During code review it was requested to insert an ability to handle different
kinds of peer id, in order to be able to add a Global Title in the future.
Add this, but only in the publicly visible API. For osmo-hlr internal code, I
intend to push implementing this into the future, when a different peer
identification actually gets introduced.
This way we don't need to implement it now in all osmo-hlr code paths (save
time now), but still make all API users aware that this type may be extended in
the future.
Change-Id: Ide9dcdca283ab989240cfc6e53e9211862a199c5
|
|
These are seemingly orthogonal changes in one patch, because they are in fact
sufficiently intertwined that we are not willing to spend the time to separate
them. They are also refactoring changes, unlikely to make sense on their own.
** lu_fsm:
Attempting to make luop.c keep state about incoming GSUP requests made me find
shortcomings in several places:
- since it predates osmo_fsm, it is a state machine that does not strictly
enforce the order of state transitions or the right sequence of incoming
events.
- several places OSMO_ASSERT() on data received from the network.
- modifies the subscriber state before a LU is accepted.
- dead code about canceling a subscriber in a previous VLR. That would be a
good thing to actually do, which should also be trivial now that we record
vlr_name and sgsn_name, but I decided to remove the dead code for now.
To both step up the LU game *and* make it easier for me to integrate
osmo_gsup_req handling, I decided to create a lu_fsm, drawing from my, by now,
ample experience of writing osmo_fsms.
** osmo_gsup_req:
Prepare for D-GSM, where osmo-hlr will do proxy routing for remote HLRs /
communicate with remote MSCs via a proxy:
a) It is important that a response that osmo-hlr generates and that is sent
back to a requesting MSC contains all IEs that are needed to route it back to
the requester. Particularly source_name must become destination_name in the
response to be able to even reach the requesting MSC. Other fields are also
necessary to match, which were so far taken care of in individual numerous code
paths.
b) For some operations, the response to a GSUP request is generated
asynchronously (like Update Location Request -> Response, or taking the
response from an EUSE, or the upcoming proxying to a remote HLR). To be able to
feed a request message's information back into the response, we must thus keep
the request data around. Since struct osmo_gsup_message references a lot of
external data, usually with pointers directly into the received msgb, it is not
so trivial to pass GSUP message data around asynchronously, on its own.
osmo_gsup_req is the combined solution for both a and b: it keeps all data for
a GSUP message by taking ownership of the incoming msgb, and it provides an
explicit API "forcing" callers to respond with osmo_gsup_req_respond(), so that
all code paths trivially are definitely responding with the correct IEs set to
match the request's routing (by using osmo_gsup_make_response() recently added
to libosmocore).
Adjust all osmo-hlr code paths to use *only* osmo_gsup_req to respond to
incoming requests received on the GSUP server (above LU code being one of
them).
In fact, the same should be done on the client side. Hence osmo_gsup_req is
implemented in a server/client agnostic way, and is placed in
libosmo-gsupclient. As soon as we see routing errors in complex GSUP setups,
using osmo_gsup_req in the related GSUP client is likely to resolve those
problems without much thinking required beyond making all code paths use it.
libosmo-gsupclient is hence added to osmo-hlr binary's own library
dependencies. It would have been added by the D-GSM proxy routing anyway, we
are just doing it a little sooner.
** gsup_peer_id.c / osmo_ipa_name:
We so far handle an IPA unit name as pointer + size, or as just pointer with
implicit talloc size. To ease working with GSUP peer identification data, I
require:
- a non-allocated storage of an IPA Name. It brings the drawback of being
size limited, but our current implementation is anyway only able to handle
MSC and SGSN names of 31 characters (see struct hlr_subscriber).
- a single-argument handle for IPA Name,
- easy to use utility functions like osmo_ipa_name_to_str(), osmo_ipa_name_cmp(), and copying
by simple assignment, a = b.
Hence this patch adds a osmo_ipa_name in gsup_peer_id.h and gsup_peer_id.c. Heavily
used in LU and osmo_gsup_req.
Depends: libosmocore Id9692880079ea0f219f52d81b1923a76fc640566
Change-Id: I3a8dff3d4a1cbe10d6ab08257a0138d6b2a082d9
|
|
Add example scripts for the distributed GSM network:
esme_dgsm.py: connect to the SMPP port of OsmoMSC A and forward SMS to the SMPP
port of OsmoMSC B. The IP and port of OsmoMSC B is retrieved by the receiver's
MSISDN using osmo-mslookup-client.
contrib/dgsm/freeswitch_dialplan_dgsm.py: resolve the destination SIP servers
of calls with osmo-mslookup-client and bridge the calls accordingly.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Related: OS#4254
Related: OS#4255
Change-Id: I26e8dd8d9a08187fccb3e74ee91366bc24f6c608
|
|
Standalone program using libosmo-mslookup to easily integrate with programs
that want to connect services (SIP, SMS,...) to the current location of a
subscriber. Also useful for manual testing.
For a detailed overview of the D-GSM and mslookup related files, please see the
elaborate comment at the top of mslookup.c (already added in an earlier patch).
Change-Id: Ie68a5c1db04fb4dff00dc3c774a1162f5b9fabf7
|
|
Add the first actually useful lookup method to the mslookup library: multicast
DNS.
The server side is added in a subsequent commit, when the mslookup server is
implemented for the osmo-hlr program.
Use custom DNS encoding instead of libc-ares (which we use in OsmoSGSN
already), because libc-ares is only a DNS client implementation and we will
need both client and server.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I03a0ffa1d4dc1b24ac78a5ad0975bca90a49c728
|
|
mslookup is a key concept in Distributed GSM, which allows querying the current
location of a subscriber in a number of cooperating but independent core
network sites, by arbitrary service names and by MSISDN/IMSI.
Add the abstract mslookup client library. An actual lookup method (besides
mslookup_client_fake.c) is added in a subsequent patch.
For a detailed overview of this and upcoming patches, please see the elaborate
comment at the top of mslookup.c.
Add as separate library, libosmo-mslookup, to allow adding D-GSM capability to
arbitrary client programs.
osmo-hlr will be the only mslookup server implementation, added in a subsequent
patch.
osmo-hlr itself will also use this library and act as an mslookup client, when
requesting the home HLR for locally unknown IMSIs.
Related: OS#4237
Patch-by: osmith, nhofmeyr
Change-Id: I83487ab8aad1611eb02e997dafbcb8344da13df1
|
|
Change the interpreter of the python script back to python3, as it was
when the script was initially added in
Idff9d757ab956179aa41ada2a223fd9f439aafbd. In the meantime, it had been
changed to python2 to make it work with build slaves that were missing
python3, but this is not necessary anymore.
This should be merged shortly after osmo-python-tests was migrated to
python3, and the jenkins build slaves were (automatically) updated to
have the new osmo-python-tests installed.
Related: OS#2819
Depends: osmo-python-tests I3ffc3519bf6c22536a49dad7a966188ddad351a7
Change-Id: Ifbb8f8f5428657a1c2d4d6d1217f8e374be43aba
|
|
Change-Id: I0c2738d380a7e79622fb3db2062c17782555e82d
|
|
gsup_client.c is GPLv2 too, having AGPLv3 in the header file does not
make sense.
Change-Id: I3827a7c70d60137ffc654c1ca53c2652bb3df147
|
|
Despite LTE/EUTRAN using the same authentication procedure (UMTS AKA)
as 3G, there's one difference: The "operator defined" 16bit AMF field
is reduced to 15 bits, with the first bit now being used as 'separation
bit'. That bit should be '0' for 2G/3G (as it is right now) and '1'
for authentication vectores generated for authentication over
EUTRAN/EPS.
Depends: libosmocore I93850710ab55a605bf61b95063a69682a2899bb1 (OSMO_GSUP_RAT_TYPES_IE)
Change-Id: Ic766bc40f6126bb479bd0a05b0e96bec3e240008
|
|
Location Updating procedures from both CS and PS overwrite the same
last_lu_seen field of a subscriber. For upcoming D-GSM it will be important to
distinguish those, because only CS attaches qualify for MSISDN lookup.
Add column last_lu_seen_ps, and upon PS LU, do not overwrite last_lu_seen, so
that last_lu_seen now only reflects CS LU.
In the VTY, dump both LU dates distinctively.
Change-Id: Id7fc50567211a0870ac0524f6dee94d4513781ba
|
|
I added these "by accident" when implementing D-GSM related VTY tests, now
submitting them separately.
Change-Id: I92a4245cae806270b00330403cc114017ab7af53
|
|
The test doesn't do much: it encodes an Insert Subscriber Data request for the
sole purpose to ensure the msgb is allocated large enough. A bug like that is
easily avoided statically.
Also, the lu functions will get refactored soon, it doesn't make sense to me to
drag this test along.
Change-Id: I42e1c72bf4cce8034f968cd4392773bf2b643c1b
|
|
VTY transcript tests run all *.vty test scripts, and it is not so
trivial to figure out the test-db creation and cmdline to run only one
of them when debugging. Add VTY_TEST var, useful to pick one test on the
cmdline:
cd tests
make vty-test VTY_TEST=test_nodes.vty
Not all VTY tests leave files behind that match hlr_vty_test.db-*, so
make sure that make does not fail it they can't be deleted (rm -f).
Change-Id: I4ad7ddb31b2bfb668b3540cfef658417dd442375
|
|
Throughout osmo-hlr's code, the GSUP msgb allocation is duplicated as:
msgb_alloc_headroom(1024+16, 16, "foo");
Instead, use one common function to keep the magic numbers in one place.
Change-Id: I40e99b5bc4fd8f750da7643c03b2119ac3bfd95e
|
|
Instead of a switch() for each version number with identical switch cases
except for the function name, use an array of function pointers and loop.
Also print a success message after each individual version upgrade, instead of
only one in the end (see change in db_upgrade_test.ok).
Change-Id: I1736af3d9a3f02e29db836966ac15ce49f94737b
|
|
db_upgrade_test.sh:
- If an ~/.sqliterc file exists, it causes output of '-- Loading resources from
~/.sqliterc'. Use -batch option to omit that.
- To make sure that column headers are off when required, add -noheaders in
some places.
Change-Id: I279a39984563594a4a3914b2ce3d803ad9468fe8
|
|
Change-Id: I0dca7a94883bbe69151d919ae204edfff12288ab
|
|
Apply the same headers structure that we keep in most Osmocom source trees:
Keep noinst_HEADERS in include/osmocom/hlr and include them using
#include <osmocom/hlr/*.h>
The only header kept in src/ is db_bootstrap.h, because it is generated during
build time. If it was built in include/osmocom/hlr, we would need db.o to
depend on db_bootstrap.h in a different subdir, which automake can't do well.
Change-Id: Ic912fe27f545b85443c5fb713d8c3c8aac23c9ad
|
|
The db bootstrap as well as the upgrade code all execute a number of
statements, and have massive code dup around each statement execution. Instead
have one db_run_statements() that takes an array of strings and runs all.
Change-Id: I2721dfc0a9aadcc7f5ac81a1c0fa87452098996f
|
|
The osmo-hlr DB schema indicates a hlr_number column and references it as 3GPP
TS 23.008 chapter 2.4.6. However, chapter 2.4.6 refers to the "MSC number",
while the "HLR number" is chapter 2.4.7.
Taking a closer look, 2.4.6 says "The MSC number is [...] stored in the HLR",
while 2.4.7 says "The HLR number may be stored in the VLR". As quite obvious,
the HLR does not store the HLR number. This was a typo from the start.
The osmo-hlr code base so far does not use the hlr_number column at all, so we
get away with renaming the column without any effects on the code base.
However, let's rather make this a new schema version to be safe.
Change-Id: I527e8627b24b79f3e9eec32675c7f5a3a6d25440
|
|
Skip the test if the installed sqlite version is older than 3.16.2
(current version of debian 9). This prevents test failures caused by
the way we dump tables in the test, which does not work with older
versions.
This patch is a middle ground between reverting the db upgrade patch,
and spending lots of time to replace the table dumping code with
something that works with old sqlite versions to fix current build
failures in OBS.
Usually version checking is done in configure.ac, however I could not
find a good way to pass the result to testsuite.at. So I decided to
use pkg-config to do the test there.
Fixes: 5b65461d686327ead0cb2a6d8493dfa9173d376c ("add db_upgrade test")
Related: https://lists.osmocom.org/pipermail/openbsc/2019-November/013063.html
Change-Id: I348c133003a95badbd6807d1519aa669115872fb
|
|
We have a database schema upgrade path, but so far nothing that verifies that
we don't break it. It almost seems like the user data weren't important to us!?
Add a db upgrade test:
- Create a db with an .sql dump taken from a db created with an old osmo-hlr,
producing DB schema version 0.
- Run osmo-hlr --db-upgrade --db-check
- Verify that the upgrade exited successfully.
- Verify that restarting with the upgraded DB works.
If python tests are enabled, also:
- create a new database using the new osmo-hlr (just built).
- replay a VTY transcript to create subscribers as in the old snapshot.
- replay some sql modifications as done in the old snapshot.
- Get a list of sorted table names,
- a list of their sorted columns with all their properties,
- and dump the table contents in a column- and value-sorted way.
- Compare the resulting dumps and error if there are any diffs.
(This is how I found the difference in the imei column that was fixed in
I68a00014a3d603fcba8781470bc5285f78b538d0)
Change-Id: I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
|
|
Move a comment for ind_bitlen column to a separate line, so that it doesn't
show in PRAGMA_TABLE_INFO('subscriber').
An upcoming patch introduces db_upgrade_test, which dumps a sorted db schema.
In newer sqlite3 versions, a comment following a 'DEFAULT' keyword actually
shows up in the PRAGMA_TABLE_INFO() results (on my machine), but older versions
(on the build slaves) drop that comment. The ind_bitlen column is the only one
producing this odd side effect, because it is the last column and has no comma
between 'DEFAULT' and the comment.
The easiest way to get matching results across sqlite3 client versions is to
move the comment to above ind_bitlen instead of having it on the same line.
(Adding a comma doesn't work.)
Change-Id: Id66ad68dd3f22d533fc3a428223ea6ad0282bde0
|
|
This allows starting osmo-hlr to merely open the database, do upgrades if
necessary, and quit, without opening any ports.
So that no ports are opened, move the telnet VTY startup to below the database
check.
Needed for upcoming patch that introduces a db_upgrade test, in
I0961bab0e17cfde5b030576c5bc243c2b51d9dc4.
Change-Id: I1a4b3360690acd2cd3cffdadffbb00a28d421316
|
|
A subsequent commit will add a db_upgrade test, which verifies that the db
resulting from an upgrade is identical to one created from scratch in the new
version. That test currently would show a diff: an upgraded 'imei' column has
'default NULL', where a new db created in version 2 has no default value on the
imei column.
Fix the upgrade path to add an imei column without 'default NULL', so that
adding the upgrade test will result in success. The test is added in
I0961bab0e17cfde5b030576c5bc243c2b51d9dc4
Change-Id: I68a00014a3d603fcba8781470bc5285f78b538d0
|
|
Change-Id: I9237b64e5748e693a5f039c5a5554d417eed3633
|
|
Change-Id: Iee675fd498ab0867ac2411d9142358f513276182
|
|
Use three dots to avoid checking for exact commands between help and
exit, which originate from libosmocore. This avoids test failues when
we slightly change the commands, like the change from "write file" to
"write file [PATH]" in [1] that is currently causing the vty tests to
fail.
[1] libosmocore I38edcf902a08b6bd0ebb9aa6fc1a7041421af525
Change-Id: I4a964b86195141e5a50705425206f3602f908999
|
|
Previous commit changed db_test to output code names to fix issues on
some platforms (I guess due to different error code values), but somehow
this log line was not updated.
Fixes: 8b860e54be3dbb6587611a9d61f6d28b4c825697
Change-Id: I598de6f83a86d528174d3d188596314572f5d70d
|
|
Change-Id: I2a1a21aceabc20fadc5dd28985a94689d2b873a2
|
|
Change-Id: Ia64f1d9f39fe2b1fb704f7b6c4d9cce93ab708cd
|
|
Change-Id: If53bdb2e216cb98df4d5a482cbbf24096db536ed
|
