This chapter will explain how to work with the Ethereal source code. It will show you how to: get the source compile the source submit changes ... However, this chapter will not explain the source file contents in detail, such as where to find a specific functionality. This is done in .

Subversion is used to keep track of the changes made to the Ethereal source code. The Ethereal source code is stored inside Ethereal project's Subversion repository located at a server at the ethereal.com domain. To qoute the Subversion book about "What is Subversion?": Subversion is a free/open-source version control system. That is, Subversion manages files and directories over time. A tree of files is placed into a central repository. The repository is much like an ordinary file server, except that it remembers every change ever made to your files and directories. This allows you to recover older versions of your data, or examine the history of how your data changed. In this regard, many people think of a version control system as a sort of "time machine". Subversion is often abbreviated as SVN, as the command-line tools are abbreviated that way. You will find both terms with the same meaning in this book, in mailing list discussions and elsewhere. Using Ethereal's Subversion repository you can: keep your private sources uptodate with very little effort get a mail notification if someone changes the latest sources get the source files from any previous release (or any other point in time) have a quick look at the sources using a web interface see which person changed a specific piece of code ... and a lot more things related to the history of the Ethereal source code development The way Ethereal uses Subversion, it can be parted into a client and a server part. Thanks to Gerald Combs (the maintainer of the Subversion server), no user usually has to deal with the Subversion server. You will only need a Subversion client, which is available as a command-line tool for many different platforms. GUI based tools also becoming more and more available these days. For further reference about Subversion, have a look at the homepage of the Subversion project: . There is a good and free book about it available at: . Please note that the anonymous Subversion repository is separate from the main repository. It may take several minutes for committed changes to appear in the anonymous repository. XXX - be more specific here. As the Ethereal project has switched from CVS (Concurrent versioning system) to Subversion some time ago, you may still find old references to CVS in the Ethereal documentation and source files.

If you need a quick look at the Ethereal source code, you will only need a Web browser. A simple view on the latest developer version can be found at: . A comprehensive view of all source versions (e.g. including the capability to show differences between versions) is available at: . Of special interest might be the subdirectories: trunk - the very latest source files releases - the source files of all released versions

There are several ways to obtain the sources from Ethereal's Subversion server. The following ways to retrieve the Ethereal sources are sorted in decreasing actuality. If you plan to commit changes you've made to the sources, it's a good idea to keep your private source tree as actual as possible. The age mentioned in the following sections will indicate, how old the most recent change in that sources will be.

Recommended for development purposes. Age: a few minutes. You can use a Subversion client to download the source code from Ethereal's anonymous Subversion repository. The URL for the repository trunk is: . Anonymous Subversion access can make your life much easier, compared to update your source tree by using any of the zip file methods mentioned below. Subversion handles merging of changes into your personal source tree in a very comfortable and quick way. So you can update your source tree several times a day without much effort. See how to install a Subversion client. For example, to check out using the command-line Subversion client, you would type: $ svn checkout http://anonsvn.ethereal.com/ethereal/trunk ethereal The checkout has to be only done once. This will copy all the sources of the latest version (including directories) from the server to your machine. This will take some time, depending on the speed of your internet line.

Recommended for development purposes, if direct Subversion access isn't possible (e.g. because of a restrictive firewall). Age: a few minutes (same as anonymous Subversion access). The entire source tree of the Subversion repository is available via a web interface at: . You can view each revision of a particular file, as well as diffs between different revisions. You can also download individual files or entire directories.

Well, not recommended at all. Age: up to 24 hours. The Subversion server will automatically generate a snapshot of Ethereal's sourcetree every night. These snapshots can be found at: . If anonymous Subversion access isn't possible, e.g. if the connection to the server isn't possible because of a corporate firewall, the sources can be obtained by downloading this nightly snapshots. However, if you are going to maintain your sources in parallel to the "official" sources for some time, it's recommended to use the anonymour Subversion access if possible (believe it, it will save you a lot of time).

Recommended for productive purposes. Age: from days to weeks. The officially released source files can be found at: . You should use these sources if you want to build Ethereal on your platform for productive use. The differences between the released sources and the sources stored at the Subversion repository are keep on growing until the next release is done (at the release time, the released and latest Subversion repository versions are then identical again :-).

After you obtained the Ethereal sources for the first time, you might want to keep them in sync with the sources at the Subversion repository.

After the first time checkout is done, updating your sources is simply done by typing (in the Ethereal source dir): $ svn update This will only take a few seconds, even on a slow internet line. It will replace old file versions by new ones. If you and someone else have changed the same file since the last update, Subversion will try to merge the changes into your private file (this is working remarkably well).

Independant of the way you retrieve the zip file of the Ethereal sources (as is providing several ways), the way to bring the changes from the official sources into your personal source tree is identical. First of all, you will download the new zip file of the official sources the way you did it the first time. If you didn't changed anything in the sources, you could simply throw away your old sources and reinstall everything just like the first time. But be sure, that you really didn't changed anything. It might be a good idea to simply rename the "old" dir to have it around, just in case you remember later that you really did changed something before. Well, if you did change something in your source tree, you have to merge the official changes since the last update into your source tree. You will install the content of the zip file into a new directory and use a good merge tool (e.g. for Win32) to bring your personal source tree in sync with the official sources again.

The sources contains several documentation files, it's a good idea to look at these files first. So after obtaining the sources, tools and libraries, the first place to look at is doc/README.developer, here you will get the latest infos for Ethereal development for all supported platforms. It is a very good idea, to first test your complete build environment (including running and debugging Ethereal) before doing any changes to the source code (unless otherwise noted). The following steps for the first time generation differs on the two major platforms.

Run the autogen.sh script at the top-level ethereal directory to configure your build directory. ./autogen.sh ./configure make If you need to build with a GTK 1.x version, you have to use: ./configure --disable-gtk2 instead of just ./configure.

The place to look at is doc/README.win32, you will get the latest infos for generation on the win32 platforms. The next thing to do will be editing the file config.nmake to reflect your configuration. The settings in this file are well documented, so please have a look at that file. Then you should cleanup any intermediate files, which are shipped for convenience of Unix users, by typing inside the command line (cmd.exe): > nmake -f Makefile.nmake distclean After doing this, typing inside the command line (cmd.exe): > nmake -f Makefile.nmake all will start the whole Ethereal build process. After the build process successfully finished, you should find an ethereal.exe and some other files in the root directory.

An already installed Ethereal may interfere with your newly generated version in various ways. If you have any problems getting your Ethereal running the first time, it might be a good idea to remove the previously installed version first. XXX - add more info here.

See the above info on running Ethereal. XXX - add more info here.

XXX - add more info here.

As the Ethereal developers working on many different platforms, a lot of editors are used to develop Ethereal (emacs, vi, Microsoft Visual Studio and many many others). There's no "standard" or "default" development environment. There are several reasons why you might want to change the Ethereal sources: add your own new dissector change/extend an existing dissector fix a bug implement a new glorious feature :-) The internal structure of the Ethereal sources will be described in . Ask the developer mailing list before you really start a new development task. If you have an idea what you want to add/change, it's a good idea to contact the developer mailing list (see ) and explain your idea. Someone else might already be working on the same topic, so double effort can be reduced, or can give you some tips what should be thought about too (like side effects that are sometimes very hard to see).

If you have finished changing the Ethereal sources to suit your needs, you might want to contribute your changes back to the Ethereal SVN repository. You gain the following benefits by contributing your improvements back to the community: Other people who find your contributions useful will appreciate them, and you will know that you have helped people in the same way that the developers of Ethereal have helped people The developers of Ethereal might improve your changes even more, as there's always room for improvements. Or they may implement some advanced things on top of your code, which can be useful for yourself too. The maintainers and developers of Ethereal will maintain your code as well, fixing it when API changes or other changes are made, and generally keeping it in tune with what is happening with Ethereal. So if Ethereal is updated (which is done often), you can get a new Ethereal version from the website and your changes will already be included without any effort for you. The maintainers and developers of Ethereal will maintain your code as well, fixing it when API changes or other changes are made, and generally keeping it in tune with what is happening with Ethereal. There's no direct way to commit changes to the SVN repository. Only a few people are authorised to actually make changes to the source code (check-in changed files). If you want to submit your changes, you should make a diff file (a patch) and send it to the developer mailing list.

A diff file is a plain text file containing the differences between a pair of files (or multiple of such pairs). A diff file is often also called a patch, as it can be used to patch an existing source file or tree with changes from somewhere else. The Ethereal community is using patches to transfer source code changes between the authors. A patch is both readable by humans and (as it is specially formatted) by some dedicated tools. Here is a small example of a patch file (XXX - generate a better example): private_data was passed to us. */ +/* Value of -1 is reserved for "not DCE packet" in packet_info.dcetransporttype. */ #define DCE_TRANSPORT_UNKNOWN 0 #define DCE_CN_TRANSPORT_SMBPIPE 1 ]]> The plus sign at the start of a line indicates an added line, a minus sign indicates a deleted line compared to the original sources. As we always use so called "unified" diff files in Ethereal development, three unchanged lines before and after the actual changed parts are included. This will make it much easier for a merge/patch tool to find the right place(s) to change in the existing sources.

There are several ways to generate such a patch.

svn diff -u [changed_files] > svn.diff XXX - add more details

Most (if not all) of the GUI Subversion clients (RapidSVN, TortoiseSVN, ...) have a built-in "diff" feature. If you use TortoiseSVN: TortoiseSVN (to be precise subversion) keeps track of the files you have changed in the directories it controls, and will generate for you a unified diff file compiling the differences. To do so - after updating your sources from the SVN repository if needed - just right-click on the highest level directory and choose "TortoiseSVN" -> "Create patch...". You will be asked for a name and then the diff file will be created. The names of the files in the patch will be relative to the directory you have right-clicked on, so it will need to be applied on that level too. When you create the diff file, it will include any difference TortoiseSVN finds in files in and under the directory you have right-clicked on, and nothing else. This means that changes you might have made for your specific configuration - like modifying "config.nmake" so that it uses your lib directory - will also be included, and you will need to remove these lines from the diff file. It also means that only changes will be recorded, i.e. if you have created new files -say, a new packet-xxx for a new protocol dissector- it will not be included in the diff, you need to add it separately. And, of course, if you have been working separately in two different patches, the .diff file will include both topics, which is probably not a good idea.

A diff file is generated, by comparing two files or directories between your own working copy and the "official" source tree. So to be able to do a diff, you should have two source trees on your computer, one with your working copy (containing your changes), and one with the "official" source tree (hopefully the latest SVN files) from www.ethereal.com. If you have only changed a single file, you could type something like this: diff -r -u --strip-trailing-cr svn-file.c work-file.c > foo.diff To get a diff file for your complete directory (including subdirectories), you could type something like this: diff -r -u --strip-trailing-cr ./svn-dir ./working-dir > foo.diff It's a good idea to do a make distclean before the actual diff call, as this will remove a lot of temporary files which might be otherwise included in the diff. After doing the diff, you should edit the foo.diff file and remove unnecessary things, like your private changes to the config.nmake file. Option Purpose -r Recursively compare any subdirectories found. -u Output unified context. --strip-trailing-cr Strip trailing carriage return on input. This is useful for Win32 -x PAT Exclude files that match PAT. This could be something like -x *.obj to exclude all win32 object files. The diff tool has a lot options, you will get a list with: diff --help

Some tips that will make the merging of your changes into the SVN tree much more likely (and you want exactly that, don't you :-): Use the latest SVN sources, or alike. It's a good idea to work with the same sources that are used by the other developer's, this makes it usually much easier to apply your patch. For information about the different ways to get the sources, see . Update your SVN sources just before making a patch. For the same reasons as the previous point. Do a "make clean" before generating the patch. This removes a lot of unneeded intermediate files (like object files) which can confuse the diff tool generating a lot of unneeded stuff which you have to remove by hand from the patch again. Find a good descriptive filename for your patch. Think a moment to find a proper name for your patch file. Often a filename like ethereal.diff is used, which isn't really helpful if keeping several of these files and find the right one later. For example: If you want to commit changes to the datatypes of dissector foo, a good filename might be: packet-foo-datatypes.diff. Follow the Ethereal source code style guide. Ethereal runs on many platforms, and can be compiled with a number of different compilers. See for details. Just because something compiles on your platform, that doesn't mean it'll compile on all of the other platforms for which Ethereal is built. Don't put unrelated things into one large patch. A few smaller patches are usually easier to apply (but also don't put every changed line into a seperate patch :-). Remove any parts of the patch not related to the changes you want to submit. You can use a text editor for this. A common example for win32 developers are the differences in your private config.nmake file. In general: making it easier to understand and apply your patch by one of the maintainers will make it much more likely (and faster) that it will actually be applied. Please remember: you don't pay the person "on the other side of the mail" for his/her effort applying your patch!

After generating a patch of your changes, you might want to have your changes included into the SVN server. You should send an email to containing: subject: [PATCH] and a short description of your changes body: the reasons for your changes and a short description what you changed and how you changed it attachment: the patch file Don't include your patch into the mail text, as this often changes the text formatting and makes it much harder to apply your patch. When someone from the Ethereal core maintainers finds the time to look at your patch, it will be merged into the SVN repository, so the latest SVN revisions and new releases will include it :-) You might get one of the following responses from your mail: your patch is checked into the SVN repository :-) your patch is rejected (and you get a response mail like: please change xy because of ...). Possible reasons: you didn't followed the style guides, your code was buggy or insecure, your code does not compile on all of the supported platforms, ... So please fix the mentioned things and send a newly generated patch. you don't get any reponse to your patch (even after a few days or so). Possible reason: your patch might simply get lost, as all core maintainers were busy at that time and forgot to look at your patch. Simply send a mail asking if the patch was forgotten or if someone is still looking at it.

Sometimes you need to apply a patch to your private source tree. Maybe because you want to try a patch from someone on the developer mailing list, or you want to check your own patch before submitting. If you have problems applying a patch, make sure the line endings (CR/NL) of the patch and your source files match. XXX - the following is a collection of material and needs some clarification. Given the file "new.diff" containing a unified diff, the right way to call the patch tool depends on what the pathnames in "new.diff" look like. If they're relative to the top-level source directory - for example, if a patch to "prefs.c" just has "prefs.c" as the file name - you'd run it as: patch -p0 <new.diff If they're relative to a higher-level directory, you'd replace 0 with the number of higher-level directories in the path, e.g. if the names are "ethereal.orig/prefs.c" and "ethereal.mine/prefs.c", you'd run it with: patch -p1 <new.diff If they're relative to a subdirectory of the top-level directory, you'd run "patch" in that directory and run it with "-p0". If you run it without "-p" at all, the patch tool flattens path names, so that if you have a patch file with patches to "Makefile.am" and "wiretap/Makefile.am", it'll try to apply the first patch to the top-level "Makefile.am" and then apply the "wiretap/Makefile.am" patch to the top-level "Makefile.am" as well. At which position in the filesystem has the patch tool to be called? If the pathnames are relative to the top-level source directory, or to a directory above that directory, you'd run it in the top-level source directory. If they're relative to a subdirectory - for example, if somebody did a patch to "packet-ip.c" and ran "diff" or "svn diff" in the "epan/dissectors" directory - you'd run it in that subdirectory. It is preferred that people NOT submit patches like that - especially if they're only patching files that exist in multiple directories, such as "Makefile.am". One other thing to note - "cvs diff" produces output that at least some versions of "patch" can't handle; you'd get something such as from "cvs diff -c", and something similar from "cvs diff -u", and "patch", unfortunately, would use the "diff -c" or "diff -u" line and try to patch "dlnames.c" in the directory you're in, rather than in the "missing" subdirectory. For "cvs diff -c" or "cvs diff -u" diffs, there's a Python script "cvsdiff-fix.py" in the "tools" directory in the Ethereal source tree; it will fix up those lines in "cvs diff" output. It reads its standard input by default, or can be given a file name on the command line, and writes to the standard output, so if you're typing at a command interpreter that does piping, you could do something such as python tools/cvsdiff.py patchfile | patch -p0 - to use "patchfile". (You might be able to leave the "python" out of the command line on many UN*Xes.) "svn diff" doesn't produce a "diff -c" or "diff -u" line, so its output doesn't have that problem. Regular "diff -c" or "diff -u" output also shouldn't have that problem. XXX - add some more details and do some cleanup.

The "usual" way to commit new files is described in . However, the following might be of interest for the "normal" developer as well. This action is only possible/allowed by the ethereal core developers who have write access to the Subversion repository. It is put in here, to have all information in one place. If you (as a core developer) need to add a file to the SVN repository, then you need to perform the following steps: Add the Ethereal boilerplate to the new file(s). Add a line to each new file, containing the following text (case is important, so don't write ID or id or iD): $Id$ Add the new file(s) to the repository: $ svn add new_file Set the line ending property to "native" for the new file(s): $ svn propset svn:eol-style native new_file Set version keyword to "Id" for the new file(s): $ svn propset svn:keywords Id new_file Commit your changes, including the added file(s). $ svn commit new_file other_files_you_modified Don't forget a brief description of the reason for the commit, so other developers don't need to read the diff in order to know what has changed.

Delivering binary packages, makes it much easier for the end-users to install Ethereal on their target system. This section will explain how the binary packages are made.

XXX - don't know how to do

XXX - don't know how to do

The "Nullsoft Install System" is a free installer generator for win32 based systems, instructions how to install it can be found in . NSIS is script based, you will find the Ethereal installer generation script at: packaging/nsis/ethereal.nsi. You will probably have to modify the config.nmake file to specify where the NSIS binaries are installed and wether to use the modern UI (which is recommended) or not. In the ethereal directory, type: > nmake -f makefile.nmake packaging to build the installer. Please be patient while the compression is done, it will take some time (a few minutes!) even on fast machines. If everything went well, you will now find something like: ethereal-setup-&EtherealCurrentVersion;.exe in the packaging/nsis directory.