TShark: Convert TTY output.

If we detect that we're writing to a TTY and that it doesn't support
UTF-8, convert our output to the current code page on UNIX/Linux or
to UTF-16LE on Windows. This helps to ensure that we don't fill users'
screens with mojibake, along with scrubbing invalid output.

Add a note about our output behavior to the TShark man page. Add a note
about the glyphs we should and shouldn't be using to utf8_entities.h.

Bug: 12393

Change-Id: I52b6dd240173b80ffb6d35b5950a46a565c97ce8
Reviewed-on: https://code.wireshark.org/review/15277
Reviewed-by: Gerald Combs <gerald@wireshark.org>
Petri-Dish: Gerald Combs <gerald@wireshark.org>
Tested-by: Petri Dish Buildbot <buildbot-no-reply@wireshark.org>
Reviewed-by: Graham Bloice <graham.bloice@trihedral.com>
Reviewed-by: Anders Broman <a.broman58@gmail.com>

1 files changed, 14 insertions, 0 deletions

diff --git a/doc/tshark.pod b/doc/tshark.pod
index fb88d53d9a..77082c7a1e 100644
--- a/doc/tshark.pod
+++ b/doc/tshark.pod
@@ -1741,6 +1741,20 @@ personal preferences file.
 
 =back
 
+=head1 OUTPUT
+
+B<TShark> uses UTF-8 to represent strings internally. In some cases the
+output might not be valid. For example, a dissector might generate
+invalid UTF-8 character sequences. Programs reading B<TShark> output
+should expect UTF-8 and be prepared for invalid output.
+
+If B<TShark> detects that it is writing to a TTY on UNIX or Linux and
+the locale does not support UTF-8, output will be re-encoded to match the
+current locale.
+
+If B<TShark> detects that it is writing to a TTY on Windows, output will be
+encoded as UTF-16LE.
+
 =head1 ENVIRONMENT VARIABLES
 
 =over 4