// WSDG Chapter Sources [[ChapterSources]] == Work with the Wireshark sources [[ChSrcIntro]] === Introduction This chapter will explain how to work with the Wireshark source code. It will show you how to: * Get the source * Compile it on your machine * Submit changes for inclusion in the official release This chapter will not explain the source file contents in detail, such as where to find specific functionality. This is done in <>. [[ChSrcGitRepository]] === The Wireshark Git repository https://git-scm.com/[Git] is used to keep track of the changes made to the Wireshark source code. The code is stored inside Wireshark project’s Git repository located at a server at the wireshark.org domain. Changes to the official repository are managed using the https://www.gerritcodereview.com/[Gerrit] code review system. Gerrit makes it easy to test and discuss changes before they are pushed to the main repository. For an overview of Gerrit see the https://code.wireshark.org/review/Documentation/intro-quick.html[Quick Introduction]. .Why Git? Git is a fast, flexible way of managing source code. It allows large scale distributed development and ensures data integrity. .Why Gerrit? Gerrit makes it easy to contribute. You can sign in with any OpenID provider and push your changes. It’s usable from both the web and command line and is integrated with many popular tools. .Git is our *third* revision control system [NOTE] ==== Wireshark originally used https://www.nongnu.org/cvs/[Concurrent Versions System] (CVS) and migrated to https://subversion.apache.org/[Subversion] in July 2004. The Subversion repository was subsequently migrated to Git in January 2014. ==== Using Wireshark’s Git repository you can: * Keep your private sources up to date with very little effort * Get a mail notification when the official source code changes * Get the source files from any previous release (or any other point in time) * Have a quick look at the sources using a web interface * See which person changed a specific piece of code * and much more [[ChSrcWebInterface]] ==== The web interface to the Git repository If you need a quick look at the Wireshark source code you can browse the most recent file versions in the master branch using Gitweb: https://code.wireshark.org/review/gitweb?p=wireshark.git;a=tree You can also view commit logs, branches, tags, and past revisions: https://code.wireshark.org/review/gitweb?p=wireshark.git ==== Git Naming Conventions Like most revision control systems, Git uses https://en.wikipedia.org/wiki/Branching_%28revision_control%29[branching] to manage different copies of the source code and allow parallel development. Wireshark uses the following branch naming conventions: * _master_: Main feature development and odd-numbered “development” releases. * _master-x.y_: Stable release maintenance. For example, master-3.0 is used to manage the 3.0.x official releases. Tags for major releases and release candidates consist of a “v” followed by a version number such as “v3.0.1” or “v3.0.3rc0”. Major releases additionally have a tag prefixed with “wireshark-” followed by a version number, such as “wireshark-3.0.0”. [[ChSrcObtain]] === Obtain the Wireshark sources There are several ways to obtain the sources from Wireshark’s Git repository. [TIP] .Check out from the master branch using Git. ==== Using Git is much easier than synchronizing your source tree by hand using any of the snapshot methods mentioned below. Git merges changes into your personal source tree in a very comfortable and quick way. So you can update your source tree several times a day without much effort. ==== [NOTE] .Keep your sources up to date ==== The following ways to retrieve the Wireshark sources are sorted in decreasing source timeliness. If you plan to commit changes you've made to the sources, it’s a good idea to keep your private source tree as current as possible. ==== The age mentioned in the following sections indicates the age of the most recent change in that set of the sources. [[ChSrcAnon]] // Retain ChSrcAnon for backward compatibility [[ChSrcGit]] ==== Git over SSH or HTTPS Recommended for development purposes. Age: a few minutes. You can use a Git client to download the source code from Wireshark’s code review system. Anyone can clone from the anonymous git URL: * {wireshark-git-anonhttp-url} If you create a Gerrit account you can clone from an authenticated URL: * {wireshark-git-ssh-url} * {wireshark-git-http-url} SSH lets you use Gerrit on the https://code.wireshark.org/review/Documentation/cmd-index.html#_server[command line]. HTTP lets you access the repository in environments that block the Gerrit SSH port (29418). At the time of this writing (early 2014) we recommend that you use the SSH interface. However, this may change as more tools take advantage of Gerrit’s HTTP REST API. The following example shows how to get up and running on the command line. See <> for information on installing and configuring graphical Git and Gerrit clients. . Sign in to {wireshark-code-review-url} using OpenID (click Register or Sign In in the upper right corner of the web page). Follow the login instructions. . In the upper right corner of the web page, click on your account name and select _Settings_. . Under _Profile_ set a username. This will be the username that you use for SSH access. For the steps below we'll assume that your username is `henry.perry`. . Select _SSH Public Keys_ and add one or more keys. You will typically upload a key for each computer that you use. . Install git-review. This is an installable package in many Linux distributions. You can also install it as a https://pypi.python.org/pypi/git-review[Python package]. (This step isn't strictly necessary but it makes working with Gerrit much easier.) To install it from Chocolatey run + -- ---- # If you have not already, install Python 3 first and then *restart* the shell. PS$> choco install -y python3 PS$> choco install -y git-review -source python # Make sure that "git-review" is present in our path (edit the version as needed). PS$>$userpath = [Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::User) PS$>$userpath += ";C:\Python37\Scripts" PS$>[Environment]::SetEnvironmentVariable("Path", $userpath, [EnvironmentVariableTarget]::User) PS$>RefreshEnv ---- -- . Now on to the command line. First, make sure `git` works: + -- ---- $ git --version ---- -- . If this is your first time using Git, make sure your username and email address are configured. This is particularly important if you plan on uploading changes. + -- ---- $ git config --global user.name "Henry Perry" $ git config --global user.email henry.perry@example.com ---- -- . Next, clone the Wireshark repository: + -- ---- $ git clone ssh://henry.perry@code.wireshark.org:29418/wireshark ---- The clone only has to be done once. This will copy all the sources (including directories) from the server to your machine and check out the latest version. This may take some time depending on the speed of your internet connection. -- . Then set up the git pre-commit hook and the push address: + -- ---- $ cd wireshark $ cp tools/pre-commit .git/hooks/ $ git config --add remote.origin.push HEAD:refs/for/master ---- This will run a few basic checks on commit to make sure that the code does not contain trivial errors. It will also warn if it is out of sync with its master copy in the tools/ directory. The change in the push address is necessary: We have an asymmetric process for pulling and pushing because of gerrit. -- . Initialize git-review. + -- ---- $ git review -s ---- This prepares your local repository for use with Gerrit, including installing the `commit-msg` hook script. -- [[ChSrcSVNWeb]] // Retain ChSrcSVNWeb for backward compatibility [[ChSrcGitWeb]] ==== Git web interface Recommended for informational purposes only, as only individual files can be downloaded. Age: a few minutes (same as anonymous Git access). The entire source tree of the Git repository is available via a web interface at {wireshark-code-browse-url}. You can view each revision of a particular file, as well as diffs between different revisions. You can also download individual files but not entire directories. [[ChSrcBuildbot]] ==== Buildbot Snapshots Recommended for development purposes, if direct Git access isn't possible (e.g. because of a restrictive firewall). Age: some number of minutes (a bit older than the Git access). The Buildbot server will automatically start to generate a snapshot of Wireshark’s source tree after a source code change is committed. These snapshots can be found at {wireshark-snapshots-url}. If Git access isn't possible, e.g. if the connection to the server isn't possible because of a corporate firewall, the sources can be obtained by downloading the Buildbot snapshots. However, if you are going to maintain your sources in parallel to the "official" sources for some time, it’s recommended to use the anonymous (or authenticated) Git access if possible (believe it, it will save you a lot of time). [[ChSrcReleased]] ==== Released sources Recommended for building pristine packages. Age: from days to weeks. The official source releases can be found at {wireshark-download-url}. You should use these sources if you want to build Wireshark on your platform for with minimal or no changes, such Linux distribution packages. The differences between the released sources and the sources in the Git repository will keep on growing until the next release is made. (At the release time, the released and latest Git repository versions are identical again :-). [[ChSrcUpdating]] === Update the Wireshark sources After you've obtained the Wireshark sources for the first time, you might want to keep them in sync with the sources at the upstream Git repository. [TIP] .Take a look at the Buildbot first ==== As development evolves, the Wireshark sources are compilable most of the time -- but not always. You should take a look at {wireshark-buildbot-url} before fetching or pulling to make sure the builds are in good shape. ==== [[ChSrcAnonUpdate]] // Retain ChSrcAnonUpdate for backward compatibility [[ChSrcGitUpdate]] ==== Update Using Git After you clone Wireshark’s Git repository you can update by running [source,sh] ---- $ git status $ git pull ---- Depending on your preferences and work habits you might want to run `git pull --rebase` or `git checkout -b my-topic-branch origin/master` instead. Fetching should only take a few seconds, even on a slow internet connection. It will update your local repository history with changes from the official repository. If you and someone else have changed the same file since the last update, Git will try to merge the changes into your private file (this works remarkably well). [[ChSrcZipUpdate]] ==== Update Using Source Archives There are several ways to download the Wireshark source code (as described in <>), but bringing the changes from the official sources into your personal source tree is identical. First of all, you will download the new `.tar.xz` file of the official sources the way you did it the first time. If you haven't changed anything in the sources, you could simply throw away your old sources and reinstall everything just like the first time. But be sure, that you really haven't changed anything. It might be a good idea to simply rename the "old" dir to have it around, just in case you remember later that you really did change something before. If you have changed your source tree, you have to merge the official changes since the last update into your source tree. You will install the content of the `.tar.xz` file into a new directory and use a good merge tool (e.g. http://winmerge.sourceforge.net/[]for Windows) to bring your personal source tree in sync with the official sources again. This method can be problematic and can be much more difficult and error-prone than using Git. [[ChSrcBuildFirstTime]] === Build Wireshark The sources contain several documentation files. It’s a good idea to read these files first. After obtaining the sources, tools and libraries, the first place to look at is _doc/README.developer_. Inside you will find the latest information for Wireshark development for all supported platforms. .Build Wireshark before changing anything [TIP] ==== It is a very good idea to first test your complete build environment (including running and debugging Wireshark) before making any changes to the source code (unless otherwise noted). ==== Building Wireshark for the first time depends on your platform. ==== Building on Unix The recommended (and fastest) way to build Wireshark is with CMake and Ninja: [source,sh] ---- # Starting from your Wireshark source directory, create a build directory # alongside it. $ cd .. $ mkdir wireshark-ninja $ cd wireshark-ninja # Assumes your source directory is named "wireshark". $ cmake -G Ninja ../wireshark $ ninja (or cmake --build .) ---- If you need to build with a non-standard configuration, you can run [source,sh] ---- $ cmake -LH ../wireshark ---- to see what options you have. ==== Windows native Follow the build procedure in <> to build Wireshark. After the build process has successfully finished, you should find a `Wireshark.exe` and some other files in the `run\RelWithDebInfo` directory. [[ChSrcRunFirstTime]] === Run generated Wireshark [TIP] .Beware of multiple Wiresharks ==== An already installed Wireshark may interfere with your newly generated version in various ways. If you have any problems getting your Wireshark running the first time, it might be a good idea to remove the previously installed version first. ==== [[ChSrcRunFirstTimeUnix]] ==== Unix-like platforms After a successful build you can run Wireshark right from the build directory. Still the program would need to know that it’s being run from the build directory and not from its install location. This has an impact on the directories where the program can find the other parts and relevant data files. In order to run the Wireshark from the build directory set the environment variable `WIRESHARK_RUN_FROM_BUILD_DIRECTORY` and run Wireshark. If your platform is properly setup, your build directory and current working directory are not in your PATH, so the command line to launch Wireshark would be: [source,sh] ---- $ WIRESHARK_RUN_FROM_BUILD_DIRECTORY=1 ./run/wireshark ---- There’s no need to run Wireshark as root user, you just won't be able to capture. When you opt to run Wireshark this way, your terminal output can be informative when things don't work as expected. [[ChSrcRunFirstTimeWin32]] ==== Windows native During the build all relevant program files are collected in a subdirectory `run\RelWithDebInfo`. You can run the program from there by launching the Wireshark.exe executable. [[ChSrcDebug]] === Debug Your Generated Wireshark [[ChSrcUnixDebug]] ==== Unix-Like Platforms You can debug using command-line debuggers such as gdb, dbx, or lldb. If you prefer a graphic debugger, you can use an IDE or debugging frontend such as Qt Creator, CLion, or Eclipse. Additional traps can be set on GLib by setting the `G_DEBUG` environment variable: [source,sh] ---- $ G_DEBUG=fatal_criticals gdb wireshark ---- See https://developer.gnome.org/glib/stable/glib-running.html[] [[ChSrcWin32Debug]] ==== Windows native You can debug using the Visual Studio Debugger or WinDbg. See the section on using the <>. [[ChSrcChange]] === Make changes to the Wireshark sources As the Wireshark developers are working on many different platforms, a lot of editors are used to develop Wireshark (emacs, vi, Microsoft Visual Studio and many, many others). There’s no "standard" or "default" development environment. There are several reasons why you might want to change the Wireshark sources: * Add support for a new protocol (a new dissector) * Change or extend an existing dissector * Fix a bug * Implement a glorious new feature The internal structure of the Wireshark sources will be described in <>. .Ask the _wireshark-dev_ mailing list before you start a new development task. [TIP] ==== If you have an idea what you want to add or change it’s a good idea to contact the developer mailing list (see <>) and explain your idea. Someone else might already be working on the same topic, so a duplicated effort can be reduced. Someone might also give you tips that should be thought about (like side effects that are sometimes very hard to see). ==== // XXX - Add a section on branching. [[ChSrcContribute]] === Contribute your changes If you have finished changing the Wireshark sources to suit your needs, you might want to contribute your changes back to the Wireshark community. You gain the following benefits by contributing your improvements: * _It’s the right thing to do._ Other people who find your contributions useful will appreciate them, and you will know that you have helped people in the same way that the developers of Wireshark have helped you. * _You get free enhancements._ By making your code public, other developers have a chance to make improvements, as there’s always room for improvements. In addition someone may implement advanced features on top of your code, which can be useful for yourself too. * _You save time and effort._ The maintainers and developers of Wireshark will maintain your code as well, updating it when API changes or other changes are made, and generally keeping it in tune with what is happening with Wireshark. So if Wireshark is updated (which is done often), you can get a new Wireshark version from the website and your changes will already be included without any effort for you. There’s no direct way to push changes to the Git repository. Only a few people are authorised to actually make changes to the source code (check-in changed files). If you want to submit your changes, you should upload them to the code review system at {wireshark-code-review-url}. This requires you to set up git as described at <>. [[ChSrcDiffWhat]] // ==== What is a diff file (a patch)? // // A https://en.wikipedia.org/wiki/Diff[diff file]is a plain text file containing the differences between a pair of files // (or a multiple of such file pairs). // // .A diff file is often also called a patch. // [TIP] // ==== // No matter what the name it can be used to patch an existing source file or tree with changes // from somewhere else. // ==== // // The Wireshark community is using patches to transfer source code changes // between the authors. // // A patch is both readable by humans and (as it is specially formatted) by // some dedicated tools. // // Here is a small example of a patch for _file.h_that // makes the second argument in cf_continue_tail()volatile. It was created using _svn diff _, // described below: // // [source,Diff] // ---- // Index: file.h // =================================================================== // --- file.h (revision 21134) // +++ file.h (revision 22401) // @@ -142,7 +142,7 @@ // * @param err the error code, if an error had occurred // * @return one of cf_read_status_t // */ // -cf_read_status_t cf_continue_tail(capture_file *cf, int to_read, int *err); // +cf_read_status_t cf_continue_tail(capture_file *cf, volatile int to_read, int *err); // // /** // * Finish reading from "end" of a capture file. // ---- // // The plus sign at the start of a line indicates an added line, a minus // sign indicates a deleted line compared to the original sources. // // We prefer to use so called "unified" diff files in Wireshark development, // three unchanged lines before and after the actual changed parts are // included. This makes it much easier for a merge/patch tool to find // the right place(s) to change in the existing sources. [[ChSrcGeneratePatch]] // ==== Generate a patch // // There are several ways to generate patches. The preferred way is to // generate them from an updated Subversion tree, since it avoids // unnecessary integration work. [[ChSrcSVNDiff]] // // ===== Using the svn command-line client // // ---- // $ svn diff [changed_files] > svn.diff // ---- // // Use the command line svn client to generate a patch in the required format // from the changes you've made to your working copy. If you leave out the // name of the changed file the svn client searches for all changes in the // working copy and usually produces a patch containing more than just the // change you want to send. Therefore you should always check the produced // patch file. // // If you've added a new file, e.g. // _packet-myprotocol.c_, you can use `svn add` to add it to your local tree before generating the patch. // Similarly, you can use `svn rm` for files that should be removed. [[ChSrcSVNGUIDiff]] // ===== Using the diff feature of the GUI Subversion clients // // Most (if not all) of the GUI Subversion clients (RapidSVN, TortoiseSVN, ...) // have a built-in "diff" feature. // // If you use TortoiseSVN: // // TortoiseSVN (to be precise Subversion) keeps track of the files you have // changed in the directories it controls, and will generate for you a // unified diff file compiling the differences. To do so - after updating // your sources from the SVN repository if needed - just right-click on the // highest level directory and choose "TortoiseSVN" -> "Create patch...". // You will be asked for a name and then the diff file will be created. The // names of the files in the patch will be relative to the directory you have // right-clicked on, so it will need to be applied on that level too. // // When you create the diff file, it will include any difference TortoiseSVN // finds in files in and under the directory you have right-clicked on, and // nothing else. This means that changes you might have made for your // specific configuration - like modifying _config.nmake_ so that it uses // your lib directory - will also be included, and you will need to remove // these lines from the diff file. It also means that only changes will be // recorded, i.e. if you have created new files -- say, a new // _packet-xxx.c_ for a // new protocol dissector -- it will not be included in the diff, you need to // add it separately. And, of course, if you have been working separately in // two different patches, the .diff file will include both topics, which is // probably not a good idea. [[ChSrcDiff]] // ===== Using the diff tool // // A diff file is generated, by comparing two files or directories between // your own working copy and the "official" source tree. So to be able to // do a diff, you should // have two source trees on your computer, one with your working copy // (containing your changes), and one with the "official" source tree // (hopefully the latest SVN files) from {wireshark-main-url}. // // If you have only changed a single file, you could type something like // this: // // ---- // $ diff -r -u --strip-trailing-cr svn-file.c work-file.c > foo.diff // ---- // // To get a diff file for your complete directory (including // subdirectories), you could type something like this: // // ---- // $ diff -N -r -u --strip-trailing-cr ./svn-dir ./working-dir > foo.diff // ---- // // It’s a good idea to run `make distclean` before the // actual diff call, as this will remove a lot // of temporary files which might be otherwise included in the diff. After // doing the diff, you should edit the _foo.diff_ file and remove unnecessary // things, like your private changes to the // _config.nmake_ file. // // // .Some useful diff options // [options="header"] // |=============== // |Option|Purpose // |-N|Add new files when used in conjunction with -r. // |-r|Recursively compare any subdirectories found. // |-u|Output unified context. // |--strip-trailing-cr|Strip trailing carriage return on input. This is useful for Win32 // // |-x PAT|Exclude files that match PAT. // This could be something like -x *.obj to exclude all win32 object files. // |=============== // // // The diff tool has a lot options; they can be listed with: // // ---- // diff --help // ---- [[ChSrcGoodPatch]] ==== Some Tips For A Good Patch Some tips that will make the merging of your changes into Git much more likely (and you want exactly that, don't you?): * _Use the latest Git sources._ It’s a good idea to work with the same sources that are used by the other developers. This usually makes it much easier to apply your patch. For information about the different ways to get the sources, see <>. * _Update your sources just before making a patch._ For the same reasons as the previous point. * _Inspect your patch carefully._ Run `git diff` and make sure you aren't adding, removing, or omitting anything you shouldn't. // * _Do a "make clean" before generating the patch._ This removes a lot of // unneeded intermediate files (like object files) which can confuse the diff // tool generating a lot of unneeded stuff which you have to remove by hand from // the patch again. // XXX - What *are* good topic names? * _Find a good descriptive topic name for your patch._ Short, specific names are preferred. _snowcone-machine-protocol_ is good, your name or your company name isn't. * _Don't put unrelated things into one large patch._ A few smaller patches are usually easier to apply (but also don't put every changed line into a separate patch). In general, making it easier to understand and apply your patch by one of the maintainers will make it much more likely (and faster) that it will actually be applied. .Please remember [NOTE] ==== Wireshark is a volunteer effort. You aren't paying to have your code reviewed and integrated. ==== [[ChSrcGoodCommitMessage]] ==== Writing a Good Commit Message When running git commit, you will be prompted to describe the change. Here are some guidelines on how to make that message actually useful to other people (and to scripts that may try to parse it): * _Provide a very brief description (under 60 characters) of the change in the first line._ If the change is specific to a single protocol, start this line with the abbreviated name of the protocol and a colon. If the change is not complete yet prefix the line with [WIP] to inform this change not to be submitted yet. This can later be removed. * _Insert a single blank line after the first line._ * _Provide a detailed description of the change in the following lines_, breaking paragraphs where needed. Limit each line to 80 characters. You can also add metadata at the bottom of the commit message which will be used by Gerrit for searching and triggering Bugzilla integration. Each line of the footer is of the form Label: value. There can be no extra line-breaks between footer lines. Wireshark currently supports the following metadata tags: .Commit message tags [options="header"] |=== |Tag|Meaning |`Change-id`|A unique hash describing the change, which is generated automatically by the git commit-msg hook which you installed during setup. This should not be changed, even when rebasing or amending a commit following code review. If you pass --no-verify to git commit you will have to add this line yourself. |`Bug`|Make Gerrit automatically add a comment and close the given bug number when the commit is merged. For use when the change does fully fix the issue. |`Ping-Bug`|Make Gerrit just add a comment to the referenced bug. For use when the change is related but does not fully fix the issue. |=== [NOTE] ==== The `Bug` and `Ping-Bug` tags are particularly useful if a capture file has been provided via a Bugzilla entry in parallel with the change. All non-trivial fixes to dissectors should try to do this. ==== Putting all that together, we get the following example: [source] ---- MIPv6: fix dissection of Service Selection Identifier APN field is not encoded as a dotted string so the first character is not a length Bug: 10323 Change-Id: Ia62137c785d505e9d0f1536a333b421a85480741 ---- [[ChSrcCodeRequirements]] ==== Code Requirements The core maintainers have done a lot of work fixing bugs and making code compile on the various platforms Wireshark supports. To ensure Wireshark’s source code quality, and to reduce the workload of the core maintainers, there are some things you should think about _before_ submitting a patch. .Pay attention to the coding guidelines [WARNING] ==== Ignoring the code requirements will make it very likely that your patch will be rejected. ==== * _Follow the Wireshark source code style guide._ Just because something compiles on your platform, that doesn't mean it'll compile on all of the other platforms for which Wireshark is built. Wireshark runs on many platforms, and can be compiled with a number of different compilers. See <>for details. * _Submit dissectors as built-in whenever possible._ Developing a new dissector as a plugin is a good idea because compiling and testing is quicker, but it’s best to convert dissectors to the built-in style before submitting for check in. This reduces the number of files that must be installed with Wireshark and ensures your dissector will be available on all platforms. + This is no hard-and-fast rule though. Many dissectors are straightforward so they can easily be put into "the big pile", while some are ASN.1 based which takes a different approach, and some multiple source file dissectors are more suitable to be placed separately as plugins. * _Ensure Wireshark Git Pre-Commit Hook is in the repository._ In your local repository directory, there will be a .git/hooks/ directory, with sample git hooks for running automatic actions before and after git commands. You can also optionally install other hooks that you find useful. + In particular, the pre-commit hook will run every time you commit a change and can be used to automatically check for various errors in your code. The sample git pre-commit hook simply detects whitespace errors such as mixed tabs and spaces; to install it just remove the .sample suffice from the existing pre-commit.sample file. + Wireshark provides a custom pre-commit hook which does additional Wireshark-specific API and formatting checks, but it might return false positives. If you want to install it, copy the pre-commit file from the tools directory (cp ./tools/pre-commit .git/hooks/) and make sure it is executable or it will not be run. + If the pre-commit hook is preventing you from committing what you believe is a valid change, you can run git commit --no-verify to skip running the hooks. Warning: using --no-verify avoids the commit-msg hook, and thus will not automatically add the required Change-ID to your commit. In case you are not updating an existing patch you may generate a Change-ID by running git review -i (or git commit --amend if don't use git review). + Additionally, if your system supports symbolic links, as all UNIX-like platforms do, you can use them instead of copying files. Running ln -s ./tools/pre-commit .git/hooks creates a symbolic link that will make the hook to be up-to-date with the current master. The same can be done for commit-msg script. * _Fuzz test your changes!_ Fuzz testing is a very effective way to automatically find a lot of dissector related bugs. You'll take a capture file containing packets affecting your dissector and the fuzz test will randomly change bytes in this file, so that unusual code paths in your dissector are checked. There are tools available to automatically do this on any number of input files, see: {wireshark-wiki-url}FuzzTesting for details. [[ChSrcUpload]] ==== Uploading your changes When you're satisfied with your changes (and obtained any necessary approval from your organization) you can upload them for review at {wireshark-code-review-url}. This requires a Gerrit Code Review account as described at <>. Changes should be pushed to a https://code.wireshark.org/review/Documentation/user-upload.html#push_create[magical "refs/for" branch] in Gerrit. For example, to upload your new Snowcone Machine Protocol dissector you could push to refs/for/master with the topic "snowcone-machine": [source,sh] ---- $ git push ssh://my.username@code.wireshark.org:29418/wireshark HEAD:refs/for/master/snowcone-machine ---- The username `my.username` is the one which was given during registration with the review system. If you have `git-review` installed you can upload the change with a lot less typing: [source,sh] ---- # Note: The "-f" flag deletes your current branch. $ git review -f ---- You can push using any Git client. Many clients have support for Gerrit, either built in or via an additional module. The Change-Id is very relevant in the review process, since it's the key used to identify one change. See the https://gerrit-review.googlesource.com/Documentation/user-changeid.html[Gerrit manual] for more details. You might get one of the following responses to your patch request: * Your patch is checked into the repository. Congratulations! * You are asked to provide additional information, capture files, or other material. If you haven't fuzzed your code, you may be asked to do so. * Your patch is rejected. You should get a response with the reason for rejection. Common reasons include not following the style guide, buggy or insecure code, and code that won't compile on other platforms. In each case you'll have to fix each problem and upload another patch. * You don't get any response to your patch. Possible reason: All the core developers are busy (e.g., with their day jobs or family or other commitments) and haven't had time to look at your patch. Don't worry, if your patch is in the review system it won't get lost. If you're concerned, feel free to add a comment to the patch or send an email to the developer’s list asking for status. But please be patient: most if not all of us do this in our spare time. [[ChSrcBackport]] ==== Backporting a change When a bug is fixed in the master branch it might be desirable or necessary to backport the fix to a stable branch. You can do this in Git by cherry-picking the change from one branch to another. Suppose you want to backport change 1ab2c3d4 from the master branch to master-1.10. Using "pure Git" commands you would do the following: [source,sh] ---- # Create a new topic branch for the backport. $ git checkout -b backport-g1ab2c3d4 origin/master-1.10 # Cherry-pick the change. Include a "cherry picked from..." line. $ git cherry-pick -x 1ab2c3d4 # If there are conflicts, fix them. # Compile and test the change. $ make $ ... # OPTIONAL: Add entries to docbook/release-notes.adoc. $ $EDITOR docbook/release-notes.adoc # If you made any changes, update your commit: $ git commit --amend -a # Upload the change to Gerrit $ git push ssh://my.username@code.wireshark.org:29418/wireshark HEAD:refs/for/master-1.10/backport-g1ab2c3d4 ---- If you want to cherry-pick a Gerrit change ID (e.g. I5e6f7890) you can use `git review -X I5e6f7890` instead of `git cherry-pick` and `git review` instead of `git push` as described in the previous chapter. [[ChSrcPatchApply]] === Apply a patch from someone else Sometimes you need to apply a patch to your private source tree. Maybe because you want to try a patch from someone on the developer mailing list, or you want to check your own patch before submitting. .Beware line endings [WARNING] ==== If you have problems applying a patch, make sure the line endings (CR/LF) of the patch and your source files match. ==== [[ChSrcPatchUse]] ==== Using patch Given the file _new.diff_ containing a unified diff, the right way to call the patch tool depends on what the pathnames in _new.diff_ look like. If they're relative to the top-level source directory (for example, if a patch to _prefs.c_ just has _prefs.c_ as the file name) you’d run it as: [source,sh] ---- $ patch -p0 < new.diff ---- If they're relative to a higher-level directory, you’d replace 0 with the number of higher-level directories in the path, e.g. if the names are _wireshark.orig/prefs.c_ and _wireshark.mine/prefs.c_, you’d run it with: [source,sh] ---- $ patch -p1 < new.diff ---- If they're relative to a _subdirectory_ of the top-level directory, you’d run `patch` in _that_ directory and run it with `-p0`. If you run it without `-pat` all, the patch tool flattens path names, so that if you have a patch file with patches to _CMakeLists.txt_ and _wiretap/CMakeLists.txt_, it'll try to apply the first patch to the top-level _CMakeLists.txt_ and then apply the _wiretap/CMakeLists.txt_ patch to the top-level _CMakeLists.txt_ as well. At which position in the filesystem should the patch tool be called? If the pathnames are relative to the top-level source directory, or to a directory above that directory, you’d run it in the top-level source directory. If they're relative to a *subdirectory* -- for example, if somebody did a patch to _packet-ip.c_ and ran `diff` or `git diff` in the _epan/dissectors_ directory -- you’d run it in that subdirectory. It is preferred that people *not* submit patches like that, especially if they're only patching files that exist in multiple directories such as _CMakeLists.txt_. [[ChSrcBinary]] === Binary Packaging Delivering binary packages makes it much easier for the end-users to install Wireshark on their target system. This section will explain how the binary packages are made. [[ChSrcVersioning]] ==== Packaging Guidelines The following guidelines should be followed by anyone creating and distributing third-party Wireshark packages or redistributing official Wireshark packages. [discrete] ===== Spelling And Capitalization Wireshark is spelled with a capital “W”, and with everything else lower case. “WireShark” in particular is incorrect. [discrete] ===== Main URL The official Wireshark project URL is https://www.wireshark.org/. [discrete] ===== Download URLs Official packages are distributed on the main web server (www.wireshark.org) and a https://www.wireshark.org/download.html#spelunking[number of download mirrors]. The canonical locations for packages are in the _all_versions_ subdirectories on each server. For example, if your packaging system links to or downloads the source tarball and you want to download from 1.na.dl.wireshark.org, use https://1.na.dl.wireshark.org/download/src/all-versions/wireshark-{wireshark-version}.tar.xz instead of https://1.na.dl.wireshark.org/download/src/wireshark-{wireshark-version}.tar.xz [discrete] ===== Artwork Logo and icon artwork can be found in the _image_ directory in the distribution. This is available online at {wireshark-code-browse-url};a=tree;f=image;hb=HEAD [discrete] ===== Licensing Wireshark is released under the GNU General Public License version 2 or later. Make sure you and your package comply with this license. [discrete] ===== Trademarks Wireshark and the “fin” logo are registered trademarks of the Wireshark Foundation. Make sure you and your package comply with trademark law. [discrete] ===== Privileges All function calls that require elevated privileges are in dumpcap. WIRESHARK CONTAINS OVER THREE MILLION LINES OF SOURCE CODE. DO NOT RUN THEM AS ROOT. Warnings are displayed when Wireshark and TShark are run as root. There are two <> on non-Windows systems that affect the privileges a normal user needs to capture traffic and list interfaces: -DDUMPCAP_INSTALL_OPTION=capabilities:: Install dumpcap with cap_net_admin and cap_net_raw capabilities. Linux only. -DDUMPCAP_INSTALL_OPTION=suid:: Install dumpcap setuid root. These are necessary for non-root users to be able to capture on most systems, e.g. on Linux or FreeBSD if the user doesn't have permissions to access /dev/bpf*. Setcap installation is preferred over setuid on Linux. If `-DDUMPCAP_INSTALL_OPTION=capabilities` is used it will override any setuid settings. The `-DENABLE_CAP` option is only useful when dumpcap is installed setuid. If it is enabled dumpcap will try to drop any setuid privileges it may have while retaining the `CAP_NET_ADMIN` and `CAP_NET_RAW` capabilities. It is enabled by default, if the Linux capabilities library (on which it depends) is found. Note that enabling setcap or setuid installation allows packet capture for ALL users on your system. If this is not desired, you can restrict dumpcap execution to a specific group or user. The following two examples show how to restrict access using setcap and setuid respectively: [source,sh] ---- # groupadd -g packetcapture # chmod 750 /usr/bin/dumpcap # chgrp packetcapture /usr/bin/dumpcap # setcap cap_net_raw,cap_net_admin+ep /usr/bin/dumpcap # groupadd -g packetcapture # chgrp packetcapture /usr/bin/dumpcap # chmod 4750 /usr/bin/dumpcap ---- [discrete] ===== Customization Custom version information can be added by running `tools/make-version.pl`. If your package contains significant changes we recommend that you use this to differentiate it from official Wireshark releases. [source, sh] ---- $ tools/make-version.pl --set-release --untagged-version-extra=-{vcsinfo}-FooCorp --tagged-version-extra=-FooCorp ---- See `tools/make-version.pl` for details. The Git version corresponding to each release is in _version.h_. It's defined as a string. If you need a numeric definition, let us know. If you have a question not addressed here, please contact {wireshark-dev-list-email}. [[ChSrcDeb]] ==== Debian: .deb Packages The Debian Package is built using dpkg-buildpackage, based on information found in the source tree under _debian_. See https://www.debian.org/doc/manuals/maint-guide/build.en.html for a more in-depth discussion of the build process. In the wireshark directory, type: [source,sh] ---- $ dpkg-buildpackage -b -us -uc -jauto ---- to build the Debian Package. [[ChSrcRpm]] ==== Red Hat: .rpm Packages You can build an RPM package using the `rpm-package` target. The package version is derived from the current git HEAD, so you must build from a git checkout. The package is built using https://rpm.org/[rpmbuild], which comes as standard on many flavours of Linux, including Red Hat, Fedora, and openSUSE. The process creates a clean build environment in _$\{CMAKE_BINARY_DIR}/packaging/rpm/BUILD_ each time the RPM is built. The settings that control the build are in _$\{CMAKE_SOURCE_DIR}/packaging/rpm/wireshark.spec.in_. The generated SPEC file contains CMake flags and other settings for the RPM build environment. Many of these come from the parent CMake environment. Notable ones are: * _\_prefix_ is set to _CMAKE_INSTALL_PREFIX_. By default this is _/usr/local_. Pass `-DCMAKE_INSTALL_PREFIX=/usr` to create a package that installs into _/usr_. * Whether or not to create the “wireshark-qt” package (`-DBUILD_wireshark`). * Lua, c-ares, nghttp2, and other library support (`-DENABLE_...`). * Building with Ninja (`-G Ninja`). In your build directory, type: [source,sh] ---- $ ninja rpm-package # ...or, if you're using GNU make... $ make rpm-package ---- to build the binary and source RPMs. When it is finished there will be a message stating where the built RPM can be found. .This might take a while [TIP] ==== This creates a tarball, extracts it, compiles Wireshark, and constructs a package. This can take quite a long time. You can speed up the process by using Ninja. If you're using GNU make you can add the following to your `~/.rpmmacros` file to enable parallel builds: ---- %_smp_mflags -j %(grep -c processor /proc/cpuinfo) ---- ==== Building the RPM package requires quite a few packages and libraries including GLib, `gcc`, `bison`, `flex`, Asciidoctor, and Qt development tools such as `uic` and `moc`. The required Qt packages can usually be obtained by installing the _qt5-devel_ package. For a complete list of build requirements, look for the “BuildRequires” lines in _packaging/rpm/wireshark.spec.in_. [[ChSrcOSX]] ==== macOS: .dmg Packages The macOS Package is built using macOS packaging tools, based on information found in the source tree under _packaging/macosx_. It requires Asciidoctor and https://pypi.org/project/dmgbuild/[dmgbuild]. In your build directory, type: [source,sh] ---- $ make dmg_package ---- to build the macOS Package. [[ChSrcNSIS]] ==== Windows: NSIS .exe Installer The _Nullsoft Install System_ is a free installer generator for Windows systems. Instructions on installing it can be found in <>. NSIS is script based. You can find the main Wireshark installer generation script at _packaging/nsis/wireshark.nsi_. When building with CMake you must first build the _nsis_package_prep_ target, followed by the _nsis_package_ target, e.g. [source,cmd] ---- > msbuild /m /p:Configuration=RelWithDebInfo nsis_package_prep.vcxproj > msbuild /m /p:Configuration=RelWithDebInfo nsis_package.vcxproj ---- Splitting the packaging projects in this way allows for code signing. [TIP] .This might take a while ==== Please be patient while the package is compressed. It might take some time, even on fast machines. ==== If everything went well, you will now find something like: _wireshark-setup-{wireshark-version}.exe_ in the _packaging/nsis_ directory in your build directory. [[ChSrcPortableApps]] ==== Windows: PortableApps .paf.exe Package _PortableApps.com_ is an environment that lets users run popular applications from portable media such as flash drives and cloud drive services. Install the _PortableApps.com Platform_. Install for “all users”, which will place it in `C:\PortableApps`. Add the following apps: - NSIS Portable (Unicode) - PortableApps.com Installer - PortableApps.com Launcher - PortableApps.com AppCompactor When building with CMake you must first build the _nsis_package_prep_ target (which takes care of general packaging dependencies), followed by the _portableapps_package_ target, e.g. [source,cmd] ---- > msbuild /m /p:Configuration=RelWithDebInfo nsis_package_prep.vcxproj > msbuild /m /p:Configuration=RelWithDebInfo portableapps_package.vcxproj ---- [TIP] .This might take a while ==== Please be patient while the package is compressed. It might take some time, even on fast machines. ==== If everything went well, you will now find something like: _WiresharkPortable_{wireshark-version}.paf.exe_ in the _packaging/portableapps_ directory. [[ChSrcMimeTypes]] === Mime Types Wireshark uses various mime-types for dragging dropping as well as file formats. This chapter gives an overview over all the mimetypes being used, as well as the data format in which data has to be provided for each individual mimetype. If not otherwise stated, the data is encoded as a Json Object. ==== Display Filter **MimeType**: application/vnd.wireshark.displayfilter Display filters are being dragged and dropped by utilizing this mime type. [source,json] ---- { "filter": "udp.port == 8080", "field": "udp.port", "description": "UDP Port" } ---- ==== Coloring Rules **MimeType**: application/vnd.wireshark.coloringrules Coloring Rules are being used for dragging and dropping color rules inside the coloring rules dialog. [source,json] ---- { "coloringrules" : [ { "disabled": false, "name": "UDP Ports for 8080", "filter": "udp.port == 8080", "foreground": "[0x0000, 0x0000, 0x0000]", "background": "[0xFFFF, 0xFFFF, 0xFFFF]" } ] } ---- ==== Filter List **MimeType**: application/vnd.wireshark.filterlist *_Internal Use only_* - used on the filter list for moving entries within the list ==== Column List **MimeType**: application/vnd.wireshark.columnlist *_Internal Use only_* - used on the column list for moving entries within the list // End of WSDG Chapter Sources // vim: set syntax=asciidoc: