| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Things to make configurable:
* IMSI and matching MSISDN ranges (start + count)
* Timeouts and number of MS (which should be related)
Change-Id: I5be2a6b4d3d21bf48625624b9e2cccb33765fe39
|
|
Related: OS#2204
Change-Id: If2844d78b52eda59c1f6d9e18363b858f2b5f007
|
|
Change-Id: I1a726049d1e4a95f1145dcf1f654c63503ceb78a
|
|
Since tests running a single MS and several of them are really similar,
generic code handling the test for N concurrent iperf3 sessions is
writen an moved to testlib, and existing and new test use that code.
Change-Id: I57799fde49e6845379cca332eec0a1f70aaecd2b
|
|
Change-Id: I6ff6bef14feb535d98ca41b9788700d699e1ef1e
|
|
Since the modem iface and the GGSN iface are on the same host/netns,
it's really difficult to conveniently test data plane without getting
routing loops. As a result, either GGSN or modem iface must be moved to
a different namespace. The decision after a few discussions was finally
to move modem interfaces to a different netns.
Expected setup:
* ofono is patched to avoid removing modem if it detects
through udev that its net iface was removed (due to for instance, net
iface being moved to another netns and thus not being reachable anymore
by systemd-udev process running in root netns).
* After ofono is started (and successfully configured all the modems and
detected its net ifaces through syfs/udev), script "modem-netns-setup.py
start" which creates a netns for each modem, naming it after its usb
path ID. net ifaces for that modem are moved into its netns.
* Modem is configured to use 802-3 data format, and as a result the net
iface is configured through DHCP (DHCP req only replied AFTER pdp ctx is
activated!).
* Since osmo-gsm-tester knowns the modem USB path ID (available in
resources.conf), it can run required steps (ifup, DHCP) to configure the
interface. The interface name is provided by ofono to osmo-gsm-tester.
* As a result, any process willing to transmit data through the modem
must be in the modem netns.
Related: OS#2308
Change-Id: Icb06bdfcdd37c797be95ab5addb28da2d9f6681c
|
|
It seems for not yet clear reasons the MS require some time after the
PDCH channels have been activated again to use them reliably. If no
sleep is used between call hangup and gprs activate pdp ctx, the MS
fails to activate the pdp ctx due to QMI error respone to the "Start
network" requested.
Related: OS#2582
Change-Id: I73b51c31309ac4c28c64ed7eb7c8c649e535aa22
|
|
Change-Id: I279e78e741b3ca841cb8508db40874f3c17b0785
|
|
Change-Id: I9b6944516c524a0c11a47aec6839f8ee9d571661
|
|
When first suites were added, osmo-nitb was used. Then new tests using
regular split components were added with "aoip_" prefix. At some point
it was clear that osmo-nitb was being deprecated so new tests for split
components were added without any prefix, as they are expected to be the
default one. Since most current and future development is going to be done
for split components, as well as new tests added, it makes sense to move
the few old testsuites using osmo-nitb to have all "nitb_" prefix, while
keeping the split component tests without prefix as it's the regular
network topology.
Change-Id: Idea2e053d337548e0e9b1b47441dbb262124f909
|
|
Now that we support modifiers in scenario files, we don't need to
duplicate tests and testsuites to dynamically set trx configuration at
run time. It can be done more easily with scenario modifiers.
Change-Id: I80c441bb5b98d5d2e95d4c6ae1efab3e5f3c40d9
|
|
answer format
Since osmo-hlr.git I1d09fab810a6bb9ab02904de72dbc9e8a414f9f9, USSD
processing has been moved to osmo-hlr and prefix routing needs to be
configured, otherwise USSD requested are answered back to the MS with an
error "ss-NotAvailable (18)".
mesage received for extension resolution (*#100#) had its format change,
so update to new format (carriage return at the end was dropped).
Change-Id: I5f75eff0df2d20d7f9b39e69184cffb0e635713d
|
|
Change-Id: If943ae0fb4e56787df02b09dea7387cb13eb9db3
|
|
Since sys.path is modified idynamically to load modules from "lib" subdir of each suite, from python env point of view all those modules share a namespace. As a result, there can be name collisions.
If a name collision appears (eg test1 loads "testlib.py" and test2
afterwards also loads its own "testlib.py"), then python interpreter
thinks the testlib.py module is already loaded, so test2 ends up using
"testlib.py" from test1.
The way to solve this is to make suite local
modules to live only through the scope of the suite, and unload the
modules once the suite is finished.
Change-Id: I4efe815f85bc4ec2ca91aa9c2d3a369048f21571
|
|
Change-Id: I8afc446d4291810c63bfaeabb7167113667dd8d1
|
|
Two different test suites are created since we want to run them with
different HW, because some HW support one type of PDCH channels, and
other HW supports the other one.
Change-Id: Id5e61eaff39ac7a6585dc7de2aeb2469dd2ce726
|
|
Change-Id: I7306e62ad4c4294370690cf9006871f798a7b9ec
|
|
Change-Id: I897757a21fee516ce96808a180121b09f06a0fea
|
|
This way we can have several tests sharing similar logic.
Change-Id: I5cfcc56970380a7c5400186fac5f504263d63c5f
|
|
osmo-msc now requires osmo-mgw and it's not expected to work properly
anymore with old mgcp code.
Change-Id: I93033f7b6133a4914f12f37511ad870b3dea3201
|
|
Change-Id: Ieb5656f3ca93f4343441d90e30fbe904382c44e3
|
|
This reverts commit 4a22ac7d2c0b24dba1939760121381e83827aa57.
Issue has been fixed in OS#2354, osmo-msc
1e67fea7ba5c6336066b78f98a28ab33b05c36c4.
Change-Id: I83d857c639db35abcd05bc87db9962d092c10eca
|
|
osmo-ggsn requires a link-local IPv6 address to be added to the
tun interface, otherwise the apn will not be configured correctly and it
won't be able to allocate addresses from the ipv6 pool later on.
Some OS don't support autoconfiguring link-local IPv6 addresses when the
interface is brought up (some linux versions are known to fail at it).
This is the case for our Prod osmo-gsm-tester setup (running debian8
with kernel 3.16.51).
Make sure we configure correctly the interface by forcing osmo-ggsn to
set on the interface and use a specific IPv6 link-local address. This is
done by using the "ipv6 link-local" vty cmd in osmo-ggsn.
After this modification, we can re-enable ipv6 gprs context creation as
it will work in Prod setup.
Related: OS#2746
Change-Id: Ib291c02a3c57a4189f9c4b1b856109be97ad2a34
|
|
osmo-ggsn is failing to create tun ipv6 device in Prod main unit.
Afterwards, as the iface is not created, it cannot find its link-local
ip and it doesn't configure the ipv6 pool. Later on, when an ipv4v6 ctx
is requested, it fails because apn doesn't support ipv6 (because the
pool is not created).
Related: OS#2746
Change-Id: I018c525a8a3d108233740ee1376b2671fefbbb59
|
|
Change-Id: I027e3bfb8f470a811e65e718e2a8c46809fe6396
|
|
Otherwise osmo-ggsn tries to allocate one IP of each type but fails
because the other APNs are ipv4-only or ipv6-only.
Change-Id: I53baa63dc1b83616a35af182cb6f56ee3d7fe38b
|
|
Only GPRS signalling setup is supported so far by osmo-gsm-tester, thus
we don't test sending data yet here, but at least we can already test
pdp context activation.
This test will be extended to run ping once we support setting up the
GPRS data plane in osmo-gsm-tester.
Change-Id: I8695029cb7a43cd48f650c88f38b4c054da0bc6b
|
|
Change-Id: I991d0329ff068aaa7c101a5f55ad40616faac121
|
|
Change-Id: I89deaa71956131766fec8fcc3bc7381ebb0e187f
|
|
Preparation to move current Test class into its own module test from
suite module.
Change-Id: I520bd046cb09042b5567d967f951f050e4e02e85
|
|
Since Change-Id Ia2882b7ca31a3219c676986e85045fa08a425d7a, osmo-bsc
uses osmo-mgw and utilizes libosmo-mgcp-client to talk to it.
This commit fixes latest constant failures in voice suite.
Change-Id: I1dadd781a357fce33e7bde55e4bcbdaeb4633359
|
|
Change-Id: I055f2b9de56da8d956c3e4944f6600dea2ee5578
|
|
Change-Id: I72f68bc980e6421a65f7d33712a587da340698e5
|
|
Change-Id: Ib19331f9c6476ac01bf729790dfd63c56de86a89
|
|
Change-Id: Id2df68bf4fba0fb542eebdeb515a3dac6d5e575e
|
|
Change-Id: Ib402effc830db293f27a877658894e454a93a606
|
|
Change-Id: I10fd47f53509f04abe1e2a39036e5d97c781ddab
|
|
... instead of using the one from from osmo vty directly.
This way we avoid having multiple word attribute value and we can skip
using quotes in the conf files.
Change-Id: I5265cc9990dd5e99dba1f6262b3a8c597a3e958d
|
|
An issue was spotted recently in tests run in production which call
modem.connect() twice, first time to check that we are unable to connect
with a wrong KI, then that we can connect with the correct KI.
As there's no current easy way to wait for "rejected" signal, we
basically wait for some time before checking if we were unable to
connect. It seems that sometimes waiting for 30 seconds is not enough,
and then a scan() async method is still in progress when we decide to
call connect() again, which will powercycle the modem. If the scan()
method returns at that time, then we try to access interfaces of the
modem which are no longer available because the modem is powered off at
that time. This triggers an exception and test fails.
To correct way to fix this is to make sure we disable pending Scan()
dbus method before powercycling the modem and starting a new Scan()
method.
The sleep time is also increased to make sure we let enough time to
register. It seems it can take about 15 seconds to do a full scan, and
sometimes during first scan iteration the BTS is still not found,
probably because it's still starting.
Related: OS#2510
Change-Id: Ic4bb1c6b72c23cd860c33bee7851bca3d0ac0e1b
|
|
This suite aims testing different authentication and encryption setups.
Change-Id: I5816ecc19a818e5b821fbc6272c9f37f9650ae10
|
|
A test case checked in that test is known to be failing at the moment
due to missing correct implementation bits for AoIP software.
Comment it in a separate commit so that is easy to revert it once we
have it implemented.
See OsmoSMSC #2354 for more information.
Change-Id: I12f8dbe1d6dc9a554438fda1e38b2e5e3245c320
|
|
esme: Add several bits to handle logic required by tests:
- Allow specifying the mode used to send an sms
- Add a parameter to ask to receive a Delivery receipt for that message
- Add sms_send_wait_resp API, which waits until the response message for
a given smpp message is received when sending an sms.
- Add receipt_was_received API, which together with
message_received_handler maintains state of the delivery receipts we
asked for and were still not received. However, the check needs to be
disabled for now because OsmoNITB doens't seem to be sending stuff
properly, see OsmoNITB #2353.
- On message_received_handler, also print alert_notification messages,
to show that there's actually a bug in OsmoNITB, see #2352.
Move old esme_ms_sms to esme_ms_sms_transaction, and explicitly state
that we are using that mode. On the same test, we can now enable the
part which asserts that sending an SMS to an msisdn with unknown
destination triggers an error. The issue was mainly that the error had
to come from the SMSC server response, not from the sent message, so
we have to wait for the response to have the failure triggered.
Finally, add esme_ms_sms_storeforward, which tests features for sms sent
using that mode, and uses the APIs described above.
Change-Id: Ia2c0c325fee14143deca8310312fc530cd9ce92e
|
|
As defined in [1], the different related actors are implemented in this
commit: ESME and SMSC.
SMSC: In Osmocom, the SMSC is currently implemented inside the NITB or
the MSC. A new Smsc abstract class is created to shared code between the
NITB and the MSC, and also makes it easier for later when the SMSC is
splitted. ESMEs can be dynamically added to its configuration in a
similar way to how the BTSs are added.
ESME: A new class Esme is created which can be used by tests to control
an ESME to interact with the SMSC. The ESME functionalities are
implemented using python-smpplib. Required version of this library is at
least 43cc6f819ec76b2c0a9d36d1d439308634716227, which contains support
for python 3 and some required features to poll the socket.
This commit already contains a few tests which checks different
features and tests the API. Extending tested features or scenarios can be
later done quite easily.
The tests are not enabled by default right now, because there are several
of them in a suite and the ip_address resources are not freed after every
tests which ends up in the suite failing due to missing reserved
resources. All the tests run alone work though. When the issue is fixed
they can then be added to the default list of tests to be run.
[1] http://opensmpp.org/specs/SMPP_v3_4_Issue1_2.pdf
Change-Id: I14ca3cb009d6d646a449ca99b0200da12085c0da
|
|
Change-Id: I35d4cb0d173eec240bccc1f3a5965a774b7b3506
|
|
A new mcc_mnc parameter is now optionally passed to connect() in order
to manually register to a specific network with a given MCC+MNC pair.
If no parameter is passed (or None), then the modem will be instructed
to attempt an automatic registration with any available network which
permits it.
We get the MCC+MNC parameter from the MSC/NITB and we pass it to the
modem object at connect time as shown in the modified tests. Two new
simple tests to check network registration is working are added in this
commit.
Ofono modems seem to be automatically registering at some point after
they are set Online=true, and we were actually using that 'feature'
before this patch. Thus, it is possible that a modem quickly becomes
registered, and we then check so before starting the scan+registration
process, which can take a few seconds.
The scanning method can take a few seconds to complete. To avoid
blocking in the dbus ofono Scan() method, this commit adds some code to
make use of glib/gdbus async methods, which are not yet supported
directly by pydbus. This way, we can continue polling while waiting for
the scan process to complete and we can register several modems in
parallel. When scan completes, a callback is run which attempts to
register. If no MCC+MNC was passed, as we just finished scanning the
modem should have enough fresh operator information to take good and
quick decisions on where to connect. If we have an MCC+MNC, then we check
the operator list received by Scan() method. If operator with desired
MCC+MNC is there, we register with it. If it's not there, we start
scanning() again asynchronously hoping the operator will show up in next
scan.
As scanning() and registration is done in the background, tests are
expected to call connect(), and then later on wait for the modem to
register by waiting/polling the method "modem.is_connected()". Tests
first check for the modem being connected and after with MSC
subscriber_attached(). The order is intentional because the later has to
poll through network and adds unneeded garbage to the pcap files bein
recorded.
Change-Id: I8d9eb47eac1044550d3885adb55105c304b0c15c
|
|
A bit of refactoring to fix logging and error reporting, and simplify the code.
This transmogrifies some of the things committed in
0ffb41440661631fa1d520c152be4cf8ebd4c46b "Add JUnit XML reports; refactor test
reporting", which did not fully match the code structuring ideas used in
osmo-gsm-tester. Also solve some problems present from the start of the code
base.
Though this is a bit of a code bomb, it would take a lot of time to separate
this into smaller bits: these changes are closely related and resulted
incrementally from testing error handling and logging details. I hope it's ok.
Things changed / problems fixed:
Allow only a single trial to be run per cmdline invocation: unbloat trial and
suite invocation in osmo-gsm-tester.py.
There is a SuiteDefinition, intended to be immutable, and a mutable SuiteRun.
SuiteDefinition had a list of tests, which was modified by the SuiteRun to
record test results. Instead, have only the test basenames in the
SuiteDefinition and create a new set of Test() instances for each SuiteRun, to
ensure that no state leaks between separate suite runs.
State leaking across runs can be seen in
http://jenkins.osmocom.org/jenkins/view/osmo-gsm-tester/job/osmo-gsm-tester_run/453/
where an earlier sms test for sysmo succeeds, but its state gets overwritten by
the later sms test for trx that fails. The end result is that both tests
failed, although the first run was successful.
Fix a problem with Origin: log.Origin allowed to be __enter__ed more than once,
skipping the second entry. The problem there is that we'd still __exit__ twice
or more, popping the Origin off the stack even though it should still remain.
We could count __enter__ recurrences, but instead, completely disallow entering
a second time.
A code path should have one 'with' statement per object, at pivotal points like
run_suites or run_tests. Individual utility functions should not do 'with' on a
central object. The structure needed is, in pseudo code:
try:
with trial:
try:
with suite_run:
try:
with test:
test_actions()
The 'with' needs to be inside the 'try', so that the exception can be handled
in __exit__ before it reaches the exception logging.
To clarify this, like test exceptions caught in Test.run(), also move suite
exception handling from Trial into SuiteRun.run_tests(). There are 'with self'
in Test.run() and SuiteRun.run_tests(), which are well placed, because these
are pivotal points in the main code path.
Log output: clearly separate logging of distinct suites and test scripts, by
adding more large_separator() calls at the start of each test. Place these
separator calls in more logical places. Add separator size and spacing args.
Log output: print tracebacks only once, for the test script where they happen.
Have less state that duplicates other state: drop SuiteRun.test_failed_ctr and
suite.test_skipped_ctr, instead add SuiteRun.count_test_results().
For test failure reporting, store the traceback text in a separate member var.
In the text report, apply above changes and unclutter to achieve a brief and
easy to read result overview: print less filler characters, drop the starting
times, drop the tracebacks. This can be found in the individual test logs.
Because the tracebacks are no longer in the text report, the suite_test.py can
just print the reports and expect that output instead of asserting individual
contents.
In the text report, print duration in precision of .1 seconds.
Add origin information and a traceback text to the junit XML result to give
more context when browsing the result XML. For 'AssertionError', add the source
line of where the assertion hit.
Drop the explicit Failure exception. We don't need one specific exception to
mark a failure, instead any arbitrary exception is treated as a failure. Use
the exception's class name as fail_type.
Though my original idea was to use raising exceptions as the only way to cause
a test failure, I'm keeping the set_fail() function as an alternative way,
because it allows test specific cleanup and may come in handy later. To have
both ways integrate seamlessly, shift some result setting into 'finally'
clauses and make sure higher levels (suite, trial) count the contained items'
stati.
Minor tweak: write the 'pass' and 'skip' reports in lower case so that the
'FAIL' stands out.
Minor tweak: pass the return code that the program exit should return further
outward, so that the exit(1) call does not cause a SystemExit exception to be
logged.
The aims of this patch are:
- Logs are readable so that it is clear which logging belongs to which test and
suite.
- The logging origins are correct (vs. parents gone missing as previously)
- A single test error does not cause following tests or suites to be skipped.
- An exception "above" Exception, i.e. SystemExit and the like, *does*
immediately abort all tests and suites, and the results for tests that were
not run are reported as "unknown" (rather than skipped on purpose):
- Raising a SystemExit aborts all.
- Hitting ctrl-c aborts all.
- The resulting summary in the log is brief and readable.
Change-Id: Ibf0846d457cab26f54c25e6906a8bb304724e2d8
|
|
Change-Id: I32079a7ed70e23d9bf170672e2220bc44d434e5d
|
|
Implement the Modem.log_info() function, use that instead of logging all modem
properties.
Tweak mo_mt_sms.py print() statements.
Pass modem object to SMS generation to inlcude the modem name as SMS token.
Change-Id: I2b17fce0b3b05594fd9038b54e5b65f5127bd0a4
|
|
It would be nicer to select the network programs as scenarios, i.e.
independently from the specifics of tests that don't care whether a NITB or a
MSC+BSC is in place. See OS#2270.
For now have a separate script for BSC+MSC+HLR operation to be able to rapidly
get the binaries to work. We might even simply drop the NITB style, in which
case we don't need to make it configurable.
Change-Id: I06d2529822f3e483ce7fffc439edfa57844e01ef
|
|
I would like to use the IP addresses also for OsmoBSC processes, so it is more
than clear now that 'nitb_iface' was the wrong naming choice.
The only distinction we may need in the future is public versus loopback
interface. To add that, we may add a trait to the 'ip_address' resource
like:
ip_address:
- addr: 10.42.42.1
type: public
- addr: 127.0.0.1
type: loopback
This way we can substitute public vs loopback addresses flexibly (e.g. using
scenarios).
Change-Id: I3ad583ae7a33f7a7bb56fe78a125f73c56a0e860
|
