Add distributed devicestate via the XMPP protocol.

(closes issue #15757)
 Reported by: Marquis
 Patches: 
       distributed_devstate-XMPP.txt uploaded by lmadsen (license 10)
 Tested by: Marquis, lmadsen, marcelloceschia
 
Review: https://reviewboard.asterisk.org/r/351/


git-svn-id: http://svn.digium.com/svn/asterisk/trunk@270519 f38db490-d61c-443f-a65b-d21fe96a405b

1 files changed, 433 insertions, 0 deletions

diff --git a/doc/distributed_devstate-XMPP.txt b/doc/distributed_devstate-XMPP.txt
new file mode 100644
index 000000000..1a8c143b4
--- /dev/null
+++ b/doc/distributed_devstate-XMPP.txt
@@ -0,0 +1,433 @@
+===============================================================================
+===
+=== XMPP PubSub Distributed Device State
+===
+=== Copyright (C) 2010, Digium, Inc.
+=== Leif Madsen <lmadsen@digium.com>
+===
+===============================================================================
+
+-------------------------------------------------------------------------------
+--- INTRODUCTION
+-------------------------------------------------------------------------------
+
+This document describes installing and utilizing XMPP PubSub events to 
+distribute device state and message waiting indication (MWI) events between
+servers. The difference between this method and OpenAIS (see 
+distributed_devstate.txt) is that OpenAIS can only be used in low latency
+networks; meaning only on the LAN, and not across the internet.
+
+If you plan on distributing device state or MWI across the internet, then you
+will require the use of XMPP PubSub events.
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Tigase Installation
+-------------------------------------------------------------------------------
+
+-- Description --
+
+Currently the only server supported for XMPP PubSub events is the Tigase open 
+source XMPP/Jabber environment. This is the server that the various Asterisk
+servers will connect to in order to distribute the events. The Tigase server can
+even be clustered in order to provide high availability for your device state;
+however, that is beyond the scope of this document.
+
+For more information about Tigase, visit their web site:
+
+    http://www.tigase.org
+
+-- Download --
+
+To download the Tigase environment, get the latest version at:
+
+    http://www.tigase.org/en/filebrowser/tigase-server
+
+-- Install --
+
+The Tigase server requires a working Java environment, including both a JRE
+(Java Runtime Environment) and a JDK (Java Development Kit), currently at least
+version 1.6.
+
+For more information about how to install Tigase, see the web site:
+
+    http://www.tigase.org/en/content/quick-start
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Tigase Configuration
+-------------------------------------------------------------------------------
+
+While installing Tigase, be sure you enable the PubSub module. Without it, the
+PubSub events won't be accepted by the server, and your device state will not be
+distributed.
+
+There are a couple of things you need to configure in Tigase before you start it
+in order for Asterisk to connect. The first thing we need to do is generate the
+self-signed certificate. To do this we use the keytool application. More
+information can be found here:
+
+    http://www.tigase.org/en/content/server-certificate
+
+-- Generating the keystore file --
+
+Generally, we need to run the following commands to generate a new keystore
+file.
+
+# cd /opt/Tigase-4.3.1-b1858/certs
+
+Be sure to change the 'yourdomain' to your domain.
+
+# keytool -genkey -alias yourdomain -keystore rsa-keystore \
+      -keyalg RSA -sigalg MD5withRSA
+
+The keytool application will then ask you for a password. Use the password
+'keystore' as this is the default password that Tigase will use to load the
+keystore file.
+
+You then need to specify your domain as the first value to be entered in the
+security certificate.
+
+What is your first and last name?
+  [Unknown]: asterisk.mydomain.tld
+What is the name of your organizational unit?
+  [Unknown]:
+What is the name of your organization?
+  [Unknown]:
+What is the name of your City or Locality?
+  [Unknown]:
+What is the name of your State or Province?
+  [Unknown]:
+What is the two-letter country code for this unit?
+  [Unknown]:
+Is CN=asterisk.mydomain.tld, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown correct?
+  [no]: yes
+
+You will then be asked for another password, in which case you must just press
+enter for the same password as Tigase will not work without them being the same.
+
+Enter key password for <mykey>
+             (RETURN if same as keystore password):
+
+
+-- Configuring init.properties --
+
+The next step is to configure the init.properties file which is used by Tigase
+to generate the tigase.xml file. Whenever you change the init.properties file
+because sure to remove the current tigase.xml file so that it will be
+regenerated at start up.
+
+# cd /opt/Tigase-4.3.1-b1858/etc
+
+Then edit the init.properties file and add the following:
+
+config-type=--gen-config-def
+--admins=admin@asterisk.mydomain.tld
+--virt-hosts=asterisk.mydomain.tld
+--debug=server
+--user-db=derby
+--user-db-uri=jdbc:derby:/opt/Tigase-4.3.1-b1858
+--comp-name-1=pubsub
+--comp-class-1=tigase.pubsub.PubSubComponent
+
+Be sure to change the domain in the --admin and --virt-hosts options. The most
+important lines are --comp-name-1 and --comp-class-1 which tell Tigase to load
+the PubSub module.
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Running Tigase
+-------------------------------------------------------------------------------
+
+You can then start the Tigase server with the tigase.sh script.
+
+# cd /opt/Tigase-4.3.1-b1858
+# ./scripts/tigase.sh start etc/tigase.conf
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Adding Buddies to Tigase
+-------------------------------------------------------------------------------
+
+At this time, Asterisk is not able to automatically register your peers for you,
+so you'll need to use an external application to do the initial registration.
+
+Pidgin is an excellent multi-protocol instant messenger application which
+supports XMPP. It runs on Linux, Windows, and OSX, and is open source. You can
+get Pidgin from http://www.pidgin.im
+
+Then add the two buddies we'll use in Asterisk with Pidgin by connecting to
+the Tigase server. For more information about how to register new buddies, see
+the Pidgin documentation.
+
+Once the initial registration is done and loaded into Tigase, you no longer need
+to worry about using Pidgin. Asterisk will then be able to load the peers into
+memory at start up.
+
+The example peers we've used in the following documentation for our two nodes
+are:
+
+server1@asterisk.mydomain.tld/astvoip1
+server2@asterisk.mydomain.tld/astvoip2
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Installing Asterisk
+-------------------------------------------------------------------------------
+
+Install Asterisk as usual. However, you'll need to make sure you have the
+res_jabber module compiled, which requires the iksemel development library.
+Additionally, be sure you have the OpenSSL development library installed so you
+can connect securly to the Tigase server.
+
+Make sure you check menuselect that res_jabber is selected so that it will
+compile.
+
+# cd asterisk-source
+# ./configure
+
+# make menuselect
+  ---> Resource Modules
+
+If you don't have jabber.conf in your existing configuration, because sure to
+copy the sample configuration file there.
+
+# cd configs
+# cp jabber.conf.sample /etc/asterisk/jabber.conf
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Configuring Asterisk
+-------------------------------------------------------------------------------
+
+We then need to configure our servers to communicate with the Tigase server. We
+need to modify the jabber.conf file on the servers. The configurations below are
+for a 2 server setup, but could be expanded for additional servers easily.
+
+The key note here is to note that the pubsub_node option needs to start with
+pubsub, so for example, pubsub.asterisk.mydomain.tld. Without the 'pubsub' your
+Asterisk system will not be able to distribute events.
+
+Additionally, you will need to specify each of the servers you need to connec to
+using the 'buddy' option.
+
+
+-- Asterisk Server 1 --
+
+[general]
+debug=no                                ;;Turn on debugging by default.
+;autoprune=yes                          ;;Auto remove users from buddy list. Depending on your
+                                        ;;setup (ie, using your personal Gtalk account for a test)
+                                        ;;you might lose your contacts list. Default is 'no'.
+autoregister=yes                        ;;Auto register users from buddy list.
+;collection_nodes=yes                   ;;Enable support for XEP-0248 for use with
+                                        ;;distributed device state.  Default is 'no'.
+;pubsub_autocreate=yes                  ;;Whether or not the PubSub server supports/is using
+                                        ;;auto-create for nodes.  If it is, we have to
+                                        ;;explicitly pre-create nodes before publishing them.
+                                        ;;Default is 'no'.
+
+[asterisk]
+type=client
+serverhost=asterisk.mydomain.tld
+pubsub_node=pubsub.asterisk.mydomain.tld
+username=server1@asterisk.mydomain.tld/astvoip1
+secret=welcome
+distribute_events=yes
+status=available
+usetls=no
+usesasl=yes
+buddy=server2@asterisk.mydomain.tld/astvoip2
+
+
+-- Asterisk Server 2 --
+
+[general]
+debug=yes				;;Turn on debugging by default.
+;autoprune=yes				;;Auto remove users from buddy list. Depending on your
+					;;setup (ie, using your personal Gtalk account for a test)
+					;;you might lose your contacts list. Default is 'no'.
+autoregister=yes			;;Auto register users from buddy list.
+;collection_nodes=yes			;;Enable support for XEP-0248 for use with
+					;;distributed device state.  Default is 'no'.
+;pubsub_autocreate=yes			;;Whether or not the PubSub server supports/is using
+					;;auto-create for nodes.  If it is, we have to
+					;;explicitly pre-create nodes before publishing them.
+					;;Default is 'no'.
+
+[asterisk]
+type=client
+serverhost=asterisk.mydomain.tld
+pubsub_node=pubsub.asterisk.mydomain.tld
+username=server2@asterisk.mydomain.tld/astvoip2
+secret=welcome
+distribute_events=yes
+status=available
+usetls=no
+usesasl=yes
+buddy=server1@asterisk.mydomain.tld/astvoip1
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Basic Testing of Asterisk with XMPP PubSub
+-------------------------------------------------------------------------------
+
+Once you have Asterisk installed with XMPP PubSub, it is time to test it out.
+
+We need to start up our first server and make sure we get connected to the XMPP
+server. We can verify this with an Asterisk console command to determine if
+we're connected.
+
+On Asterisk 1 we can run 'jabber show connected' to verify we're connected to
+the XMPP server.
+
+*CLI> jabber show connected 
+Jabber Users and their status:
+       User: server1@asterisk.mydomain.tld/astvoip1     - Connected
+----
+   Number of users: 1
+
+The command above has given us output which verifies we've connected our first
+server.
+
+We can then check the state of our buddies with the 'jabber show buddies' CLI
+command.
+
+*CLI> jabber show buddies
+Jabber buddy lists
+Client: server1@asterisk.mydomain.tld/astvoip1
+	Buddy:	server2@asterisk.mydomain.tld
+		Resource: None
+	Buddy:	server2@asterisk.mydomain.tld/astvoip2
+		Resource: None
+
+The output above tells us we're not connected to any buddies, and thus we're not
+distributing state to anyone (or getting it from anyone). That makes sense since
+we haven't yet started our other server.
+
+Now, let's start the other server and verify the servers are able to establish
+a connection between each other.
+
+On Asterisk 2, again we run the 'jabber show connected' command to make sure
+we've connected successfully to the XMPP server.
+
+*CLI> jabber show connected 
+Jabber Users and their status:
+       User: server2@asterisk.mydomain.tld/astvoip2     - Connected
+----
+   Number of users: 1
+
+And now we can check the status of our buddies.
+
+*CLI> jabber show buddies
+Jabber buddy lists
+Client: server2@scooter/astvoip2
+	Buddy:	server1@asterisk.mydomain.tld
+		Resource: astvoip1
+			node: http://www.asterisk.org/xmpp/client/caps
+			version: asterisk-xmpp
+			Jingle capable: yes
+		Status: 1
+		Priority: 0
+	Buddy:	server1@asterisk.mydomain.tld/astvoip1
+		Resource: None
+
+Excellent! So we're connected to the buddy on Asterisk 1, and we could run the
+same command on Asterisk 1 to verify the buddy on Asterisk 2 is seen.
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Testing Distributed Device State
+-------------------------------------------------------------------------------
+
+The easiest way to test distributed device state is to use the DEVICE_STATE()
+diaplan function.  For example, you could have the following piece of dialplan
+on every server:
+
+[devstate_test]
+
+exten => 1234,hint,Custom:mystate
+
+exten => set_inuse,1,Set(DEVICE_STATE(Custom:mystate)=INUSE)
+exten => set_not_inuse,1,Set(DEVICE_STATE(Custom:mystate)=NOT_INUSE)
+
+exten => check,1,NoOp(Custom:mystate is ${DEVICE_STATE(Custom:mystate)})
+
+
+Now, you can test that the cluster-wide state of "Custom:mystate" is what
+you would expect after going to the CLI of each server and adjusting the state.
+
+server1*CLI> console dial set_inuse@devstate_test
+   ...
+
+server2*CLI> console dial check@devstate_test
+    -- Executing [check@devstate_test:1] NoOp("OSS/dsp", "Custom:mystate is INUSE") in new stack
+
+Various combinations of setting and checking the state on different servers can
+be used to verify that it works as expected.  Also, you can see the status of
+the hint on each server, as well, to see how extension state would reflect the
+state change with distributed device state:
+
+server2*CLI> core show hints
+    -= Registered Asterisk Dial Plan Hints =-
+                   1234@devstate_test       : Custom:mystate        State:InUse           Watchers  0
+
+
+One other helpful thing here during testing and debugging is to enable debug
+logging.  To do so, enable debug on the console in /etc/asterisk/logger.conf.
+Also, enable debug at the Asterisk CLI.
+
+*CLI> core set debug 1
+
+When you have this debug enabled, you will see output during the processing of
+every device state change.  The important thing to look for is where the known
+state of the device for each server is added together to determine the overall
+state.
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Notes On Large Installations
+-------------------------------------------------------------------------------
+
+On larger installations where you want a fully meshed network of buddies (i.e.
+all servers have all the buddies of the remote servers), you may want some
+method of distributing those buddies automatically so you don't need to modify
+all servers (N+1) every time you add a new server to the cluster.
+
+The problem there is that you're confined by what's allowed in XEP-0060, and 
+unfortunately that means modifying affiliations by individual JID (as opposed to
+the various subscription access models, which are more flexible).
+
+See here for details:
+http://xmpp.org/extensions/xep-0060.html#owner-affiliations
+
+One method for making this slightly easier is to utilize the #exec functionality
+in configuration files, and dynamically generate the buddies via script that
+pulls the information from a database, or to #include a file which is 
+automatically generated on all the servers when you add a new node to the
+cluster.
+
+Unfortunately this still requires a reload of res_jabber.so on all the servers,
+but this could also be solved through the use of the Asterisk Manager Interface
+(AMI).
+
+So while this is not the ideal situation, it is programmatically solvable with
+existing technologies and features found in Asterisk today.
+
+-------------------------------------------------------------------------------
+
+-------------------------------------------------------------------------------
+--- Questions, Comments, and Bug Reports
+-------------------------------------------------------------------------------
+
+Please utilize the Asterisk issue tracker for all bug reports at
+https://issues.asterisk.org