Start porting our test scripts to Python. Add ctest support.

Create Python versions of our various test shell scripts. Add CMake
tests for each suite. Tests can now be run directly via test.py, via the
"test" target, or via ctest, e.g.

  ctest --verbose --jobs 3

Add a testing chapter to the Developer's Guide.

Add a way to disable ctest in dpkg-buildpackage.

Suites completed:
- capture
- clopts
- decryption
- dissection

Remaining suites:
- fileformats
- io
- mergecap
- nameres
- text2pcap
- unittests
- wslua

Change-Id: I8936e05edefc76a86b6a7a5da302e7461bbdda0f
Reviewed-on: https://code.wireshark.org/review/27134
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot
Reviewed-by: Peter Wu <peter@lekensteyn.nl>
Reviewed-by: Gerald Combs <gerald@wireshark.org>

2 files changed, 172 insertions, 0 deletions

diff --git a/docbook/developer-guide.asciidoc b/docbook/developer-guide.asciidoc
index 468e42874c..c89bd0c606 100644
--- a/docbook/developer-guide.asciidoc
+++ b/docbook/developer-guide.asciidoc
@@ -56,4 +56,6 @@ include::wsluarm.asciidoc[]
 
 include::wsdg_src/WSDG_chapter_userinterface.asciidoc[]
 
+include::wsdg_src/WSDG_chapter_tests.asciidoc[]
+
 include::common_src/GPL_appendix.asciidoc[]
diff --git a/docbook/wsdg_src/WSDG_chapter_tests.asciidoc b/docbook/wsdg_src/WSDG_chapter_tests.asciidoc
new file mode 100644
index 0000000000..249e5a416d
--- /dev/null
+++ b/docbook/wsdg_src/WSDG_chapter_tests.asciidoc
@@ -0,0 +1,170 @@
+// WSDG Chapter Setup
+
+[[ChapterTests]]
+== Wireshark Tests
+
+The Wireshark sources include a collection of Python scripts that test
+the features of Wireshark, TShark, Dumpcap, and other programs that
+accompany Wireshark.
+
+The command line options of Wireshark and its companion command line
+tools are numerous. These tests help to ensure that we don't introduce
+bugs as Wireshark grows and evolves.
+
+=== Quick Start
+
+The main testing script is `test.py`. It will attempt to test as much as
+possible by default, including packet capture. This means that you will
+probably either have to supply a capture interface (`--capture-interface
+<interface>`) or disable capture tests (`--disable-capture`).
+
+To run all tests from CMake do the following:
+* Pass `-DTEST_EXTRA_ARGS=--disable-capture` or
+  `-DTEST_EXTRA_ARGS=--capture-interface=<interface>`
+  as needed for your system.
+* Build the “test” target or run ctest, e.g. `ctest --jobs=4 --verbose`.
+
+To run all tests directly, run `test.py -p
+/path/to/wireshark-build/run-directory <capture args>`.
+
+To see a list of all options, run `test.py -h` or `test.py --help`.
+
+To see a list of all tests, run `test.py -l`.
+
+=== Test Coverage And Availability
+
+The testing framework can run programs and check their stdout, stderr,
+and exit codes. It cannot interact with the Wireshark UI. Tests cover
+capture, command line options, decryption, file format support and
+conversion, Lua scripting, and other functionality.
+
+Available tests depend on the libraries with which Wireshark was built.
+For example, some decryption tests depend on a minimum version of
+Libgcrypt and Lua tests depend on Lua.
+
+Capture tests depend on the permissions of the user running the test
+script. We assume that the test user has capture permissions on Windows
+and macOS and capture tests are enabled by default on those platforms.
+
+=== Suites, Cases, and Tests
+
+The `test.py` script uses Python's “unittest” module. Our tests are
+patterned after it, and individual tests are organized according to
+suites, cases, and individual tests. Suites correspond to python modules
+that match the pattern “suite_*.py”. Cases correspond to one or more
+classes in each module, and case class methods matching the pattern
+”test_*” correspond to individual tests. For example, the invalid
+capture filter test in the TShark capture command line options test case
+in the command line options suite has the ID
+“suite_clopts.case_tshark_capture_clopts.test_tshark_invalid_capfilter”.
+
+=== Listing And Running Tests
+
+Tests can be run via the `test.py` Python script. To run all tests,
+either run `test.py` in the directory that contains the Wireshark
+executables (`wireshark`, `tshark`, etc.), or pass the the executable
+path via the `-p` flag:
+
+[source,sh]
+----
+$ python test.py -p /path/to/wireshark-build/run
+----
+
+You can list tests by passing one or more complete or partial names to
+`tshark.py`. The `-l` flag lists tests. By default all tests are shown.
+
+[source,sh]
+----
+# List all tests
+$ python test.py -l
+$ python test.py -l all
+$ python test.py --list
+$ python test.py --list all
+
+# List only tests containing "dumpcap"
+$ python test.py -l dumpcap
+
+# List all suites
+$ python test.py --list-suites
+
+# List all suites and cases
+$ python test.py --list-cases
+----
+
+If one of the listing flags is not present, tests are run. If no names or `all` is supplied,
+all tests are run. Otherwise tests that match are run.
+
+[source,sh]
+----
+# Run all tests
+$ python test.py
+$ python test.py all
+
+# Only run tests containing "dumpcap"
+$ python test.py -l dumpcap
+
+# Run the "clopts" suite
+$ python test.py suite_clopts
+----
+
+=== Adding Or Modifying Tests
+
+Tests must be in a Python module whose name matches “suite_*.py”. The
+module must contain one or more subclasses of “SubprocessTestCase” or
+“unittest.TestCase”. “SubprocessTestCase” is recommended since it
+contains several convenience methods for running processes, checking
+output, and displaying error information. Each test case method
+whose name starts with “test_” constitutes an individual test.
+
+Success or failure conditions can be signalled using the
+“unittest.assertXXX()” or “subprocesstest.assertXXX()” methods.
+
+The “config” module contains common configuration information which has
+been derived from the current environment or specified on the command
+line.
+
+The “subprocesstest” class contains the following methods for running
+processes. Stdout and stderr is written to “<test id>.log”:
+
+startProcess:: Start a process without waiting for it to finish.
+runProcess:: Start a process and wait for it to finish.
+assertRun:: Start a process, wait for it to finish, and check its exit code.
+
+All of the current tests run one or more of Wireshark's suite of
+executables and either checks their return code or their output. A
+simple example is “suite_clopts.case_basic_clopts.test_existing_file”,
+which reads a capture file using TShark and checks its exit code.
+
+[source,python]
+----
+import config
+import subprocesstest
+
+class case_basic_clopts(subprocesstest.SubprocessTestCase):
+    def test_existing_file(self):
+        cap_file = os.path.join(self.capture_dir, 'dhcp.pcap')
+        self.assertRun((config.cmd_tshark, '-r', cap_file))
+----
+
+Program output can be checked using “subprocesstest.grepOutput”
+or “subprocesstest.countOutput”:
+
+[source,python]
+----
+import config
+import subprocesstest
+
+class case_decrypt_80211(subprocesstest.SubprocessTestCase):
+    def test_80211_wpa_psk(self):
+        capture_file = os.path.join(config.capture_dir, 'wpa-Induction.pcap.gz')
+        self.runProcess((config.cmd_tshark,
+                '-o', 'wlan.enable_decryption: TRUE',
+                '-Tfields',
+                '-e', 'http.request.uri',
+                '-r', capture_file,
+                '-Y', 'http',
+            ),
+            env=config.test_env)
+        self.assertTrue(self.grepOutput('favicon.ico'))
+----
+