// WSDG Chapter Setup

[[ChapterTests]]
== Wireshark Tests

The Wireshark sources include a collection of Python scripts that test
the features of Wireshark, TShark, Dumpcap, and other programs that
accompany Wireshark.

The command line options of Wireshark and its companion command line
tools are numerous. These tests help to ensure that we don't introduce
bugs as Wireshark grows and evolves.

=== Quick Start

Before running any tests you should build the “test-programs” target. It
is required for the “utittests” suite.

The main testing script is `test.py`. It will attempt to test as much as
possible by default, including packet capture. This means that you will
probably either have to supply a capture interface (`--capture-interface
<interface>`) or disable capture tests (`--disable-capture`).

To run all tests from CMake do the following:
* Pass `-DTEST_EXTRA_ARGS=--disable-capture` or
  `-DTEST_EXTRA_ARGS=--capture-interface=<interface>`
  as needed for your system.
* Build the “test” target or run ctest, e.g. `ctest --force-new-ctest-process -j 4 --verbose`.

On Windows, “ctest” requires a build configuration parameter, e.g.
`ctest -C RelWithDebInfo --force-new-ctest-process -j 4 --verbose`.

To run all tests directly, run `test.py -p
/path/to/wireshark-build/run-directory <capture args>`.

To see a list of all options, run `test.py -h` or `test.py --help`.

To see a list of all tests, run `test.py -l`.

=== Test Coverage And Availability

The testing framework can run programs and check their stdout, stderr,
and exit codes. It cannot interact with the Wireshark UI. Tests cover
capture, command line options, decryption, file format support and
conversion, Lua scripting, and other functionality.

Available tests depend on the libraries with which Wireshark was built.
For example, some decryption tests depend on a minimum version of
Libgcrypt and Lua tests depend on Lua.

Capture tests depend on the permissions of the user running the test
script. We assume that the test user has capture permissions on Windows
and macOS and capture tests are enabled by default on those platforms.

=== Suites, Cases, and Tests

The `test.py` script uses Python's “unittest” module. Our tests are
patterned after it, and individual tests are organized according to
suites, cases, and individual tests. Suites correspond to python modules
that match the pattern “suite_*.py”. Cases correspond to one or more
classes in each module, and case class methods matching the pattern
”test_*” correspond to individual tests. For example, the invalid
capture filter test in the TShark capture command line options test case
in the command line options suite has the ID
“suite_clopts.case_tshark_capture_clopts.test_tshark_invalid_capfilter”.

=== Listing And Running Tests

Tests can be run via the `test.py` Python script. To run all tests,
either run `test.py` in the directory that contains the Wireshark
executables (`wireshark`, `tshark`, etc.), or pass the the executable
path via the `-p` flag:

[source,sh]
----
$ python test.py -p /path/to/wireshark-build/run
----

You can list tests by passing one or more complete or partial names to
`tshark.py`. The `-l` flag lists tests. By default all tests are shown.

[source,sh]
----
# List all tests
$ python test.py -l
$ python test.py -l all
$ python test.py --list
$ python test.py --list all

# List only tests containing "dumpcap"
$ python test.py -l dumpcap

# List all suites
$ python test.py --list-suites

# List all suites and cases
$ python test.py --list-cases
----

If one of the listing flags is not present, tests are run. If no names or `all` is supplied,
all tests are run. Otherwise tests that match are run.

[source,sh]
----
# Run all tests
$ python test.py
$ python test.py all

# Only run tests containing "dumpcap"
$ python test.py -l dumpcap

# Run the "clopts" suite
$ python test.py suite_clopts
----

=== Adding Or Modifying Tests

Tests must be in a Python module whose name matches “suite_*.py”. The
module must contain one or more subclasses of “SubprocessTestCase” or
“unittest.TestCase”. “SubprocessTestCase” is recommended since it
contains several convenience methods for running processes, checking
output, and displaying error information. Each test case method
whose name starts with “test_” constitutes an individual test.

Success or failure conditions can be signalled using the
“unittest.assertXXX()” or “subprocesstest.assertXXX()” methods.

The “config” module contains common configuration information which has
been derived from the current environment or specified on the command
line.

The “subprocesstest” class contains the following methods for running
processes. Stdout and stderr is written to “<test id>.log”:

startProcess:: Start a process without waiting for it to finish.
runProcess:: Start a process and wait for it to finish.
assertRun:: Start a process, wait for it to finish, and check its exit code.

All of the current tests run one or more of Wireshark's suite of
executables and either checks their return code or their output. A
simple example is “suite_clopts.case_basic_clopts.test_existing_file”,
which reads a capture file using TShark and checks its exit code.

[source,python]
----
import config
import subprocesstest

class case_basic_clopts(subprocesstest.SubprocessTestCase):
    def test_existing_file(self):
        cap_file = os.path.join(self.capture_dir, 'dhcp.pcap')
        self.assertRun((config.cmd_tshark, '-r', cap_file))
----

Program output can be checked using “subprocesstest.grepOutput”
or “subprocesstest.countOutput”:

[source,python]
----
import config
import subprocesstest

class case_decrypt_80211(subprocesstest.SubprocessTestCase):
    def test_80211_wpa_psk(self):
        capture_file = os.path.join(config.capture_dir, 'wpa-Induction.pcap.gz')
        self.runProcess((config.cmd_tshark,
                '-o', 'wlan.enable_decryption: TRUE',
                '-Tfields',
                '-e', 'http.request.uri',
                '-r', capture_file,
                '-Y', 'http',
            ),
            env=config.test_env)
        self.assertTrue(self.grepOutput('favicon.ico'))
----

Tests can be run in parallel. This means that any files you create must
be unique for each test. “subprocesstest.filename_from_id” can be used
to generate a filename based on the current test name. It also ensures
that the file will be automatically removed after the test has run.