* Application SMS

The SMS module for asterisk was developed by Adrian Kennard, and is an implementation of the ETSI specification for landline SMS, ETSI ES 201 912, which is available from www.etsi.org. Landline SMS is starting to be available in various parts of Europe, and is available from BT in the UK. However, asterisk would allow gateways to be created in other locations such as the US, and use of SMS capable phones such as the Magic Messenger. SMS works using analogue or ISDN lines.

Background

Short Message Service (SMS), or texting is very popular between mobile phones. A message can be sent between two phones, and normally contains 160 characters. There are ways in which various types of data can be encoded in a text message such as ring tones, and small graphic, etc. Text messaging is being used for voting and competitions, and also SPAM...

Sending a message involves the mobile phone contacting a message centre (SMSC) and passing the message to it. The message centre then contacts the destination mobile to deliver the message. The SMSC is responsible for storing the message and trying to send it until the destination mobile is available, or a timeout.

Landline SMS works in basically the same way. You would normally have a suitable text capable landline phone, or a separate texting box such as a Magic Messenger on your phone line. This sends a message to a message centre your telco provides by making a normal call and sending the data using 1200 Baud FSK signaling according to the ETSI spec. To receive a message the message centre calls the line with a specific calling number, and the text capable phone answers the call and receives the data using 1200 Baud FSK signaling. This works particularly well in the UK as the calling line identity is sent before the first ring, so no phones in the house would ring when a message arrives.

Typical use with asterisk

Sending messages from an asterisk box can be used for a variety of reasons, including notification from any monitoring systems, email subject lines, etc.
Receiving messages to an asterisk box is typically used just to email the messages to someone appropriate - we email and texts that are received to our direct numbers to the appropriate person. Received messages could also be used to control applications, manage competitions, votes, post items to IRC, anything.
Using a terminal such as a magic messenger, an asterisk box could ask as a message centre sending messages to the terminal, which will beep and pop up the message (and remember 100 or so messages in its memory).

Terminology

SMS
 Short Message Service
 i.e. text messages
SMSC
 Short Message Service Centre
 The system responsible for storing and forwarding messages
MO
 Mobile Originated
 A message on its way from a mobile or landline device to the SMSC
MT
 Mobile Terminated
 A message on its way from the SMSC to the mobile or landline device
RX
 Receive
 A message coming in to the asterisk box
TX
 Transmit
 A message going out of the asterisk box

Sub address

When sending a message to a landline, you simply send to the landline number. In the UK, all of the mobile operators (bar one) understand sending messages to landlines and pass the messages to the BTText system for delivery to the landline.

The specification for landline SMS allows for the possibility of more than one device on a single landline. These can be configured with Sub addresses which are a single digit. To send a message to a specific device the message is sent to the landline number with an extra digit appended to the end. The telco can define a default sub address (9 in the UK) which is used when the extra digit is not appended to the end. When the call comes in, part of the calling line ID is the sub address, so that only one device on the line answers the call and receives the message.

Sub addresses also work for outgoing messages. Part of the number called by the device to send a message is its sub address. Sending from the default sub address (9 in the UK) means the message is delivered with the sender being the normal landline number. Sending from any other sub address makes the sender the landline number with an extra digit on the end.

Using asterisk, you can make use of the sub addresses for sending and receiving messages. Using DDI (DID, i.e. multiple numbers on the line on ISDN) you can also make use of many different numbers for SMS.

Build / installation

app_sms.c is included in the latest cvs. It lives in the asterisk source apps directory and is included in the object list (app_sms.so) in apps/Makefile.
smsq.c is a stand alone helper application which is used to send SMSs from the command line. It uses the popt library. A line for your make file is:-
smsq: smsq.c
	cc -O -o smsq smsq.c -lpopt

extensions.conf

The following contexts are recommended.
; Mobile Terminated, RX. This is used when an incoming call from the SMS arrives, with the queue (called number and sub address) in ${EXTEN}
; Running an app after receipt of the text allows the app to find all messages in the queue and handle them, e.g. email them.
; The app may be something like   smsq --process=somecommand --queue=${EXTEN}  to run a command for each received message
; See below for usage
[smsmtrx]
exten = _X.,1, SMS(${EXTEN}|a)
exten = _X.,2,System("someapptohandleincomingsms ${EXTEN}")
exten = _X.,3,Hangup

; Mobile originated, RX. This is receiving a message from a device, e.g. a Magic Messenger on a sip extension
; Running an app after receipt of the text allows the app to find all messages in the queue and handle then, e.g. sending them to the public SMSC
; The app may be something like   smsq --process=somecommand --queue=${EXTEN}  to run a command for each received message
; See below for example usage
[smsmorx]
exten = _X.,1, SMS(${EXTEN}|sa)
exten = _X.,2,System("someapptohandlelocalsms ${EXTEN}")
exten = _X.,3,Hangup
smsmtrx is normally accessed by an incoming call from the SMSC. In the UK this call is from a CLI of 080058752X0 where X is the sub address. As such a typical usage in the extensions.conf at the point of handling an incoming call is:-
exten = _X./8005875290,1,Goto(smsmtrx,${EXTEN},1)
exten = _X./_80058752[0-8]0,1,Goto(smsmtrx,${EXTEN}-${CALLERIDNUM:8:1},1)
Alternatively, if you have the correct national prefix on incoming CLI, e.g. using zaphfc, you might use:-
exten = _X./08005875290,1,Goto(smsmtrx,${EXTEN},1)
exten = _X./_080058752[0-8]0,1,Goto(smsmtrx,${EXTEN}-${CALLERIDNUM:9:1},1)
smsmorx is normally accessed by a call from a local sip device connected to a Magic Messenger. It could however by that you are operating asterisk as a message centre for calls from outside. Either way, you look at the called number and goto smsmorx. In the UK, the SMSC number that would be dialed is 1709400X where X is the caller sub address. As such typical usage in extension.config at the point of handling a call from a sip phone is:-
exten = 17094009,1,Goto(smsmorx,${CALLERIDNUM},1)
exten = _1709400[0-8],1,Goto(smsmorx,${CALLERIDNUM}-{EXTEN:7:1},1)

Using smsq

smsq is a simple helper application designed to make it easy to send messages from a command line. it is intended to run on the asterisk box and have direct access to the queue directories for SMS and for asterisk.

In its simplest form you can send an SMS by a command such as

smsq 0123456789 This is a test to 0123456789

This would create a queue file for a mobile originated TX message in queue 0 to send the text "This is a test to 0123456789" to 0123456789. It would then place a file in the /var/spool/asterisk/outgoing directory to initiate a call to 17094009 (the default message centre in smsq) attached to application SMS with argument of the queue name (0).

Normally smsq will queue a message ready to send, and will then create a file in the asterisk outgoing directory causing asterisk to actually connect to the message centre or device and actually send the pending message(s).

Using --process, smsq can however be used on received queues to run a command for each file (matching the queue if specified) with various environment variables set based on the message (see below);

smsq options:-

--help
Show help text
--usage
Show usage
--queue
 -q
 Specify a specific queue
In no specified, messages are queued under queue "0"
--da
 -d
 Specify destination address
--oa
 -o
 Specify originating address
This also implies that we are generating a mobile terminated message
--ud
 -m
 Specify the actual message
--ud-file
 -f
 Specify a file to be read for the context of the message
A blank filename (e.g. --ud-file= on its own) means read stdin. Very useful when using via ssh where command line parsing could mess up the message.
--mt
 -t
 Mobile terminated message to be generated
--mo
Mobile originated message to be generated
Default
--tx
Transmit message
Default
--rx
 -r
 Generate a message in the receive queue
--UTF-8
Treat the file as UTF-8 encoded (default)
--UCS-1
Treat the file as raw 8 bit UCS-1 data, not UTF-8 encoded
--UCS-2
Treat the file as raw 16 bit bigendian USC-2 data
--process
Specific a command to process for each file in the queue
Implies --rx and --mt if not otherwise specified.
Sets environment variables for every possible variable, and also ud, ud8 (USC-1 hex), and ud16 (USC-2 hex) for each call. Removes files.
--motx-channel
Specify the channel for motx calls
May contain X to use sub address based on queue name or may be full number
Default is Local/1709400X
--motx-callerid
Specify the caller ID for motx calls
The default is the queue name without -X suffix
--motx-wait
Wait time for motx call
Default 10
--motx-delay
Retry time for motx call
Default 1
--motx-retries
Retries for motx call
Default 10
--mttx-channel
Specify the channel for mttx calls
Default is Local/ and the queue name without -X suffix
--mtttx-callerid
Specify the callerid for mttx calls
May include X to use sub address based on queue name or may be full number
Default is 080058752X0
--mttx-wait
Wait time for mttx call
Default 10
--mttx-delay
Retry time for mttx call
Default 30
--mttx-retries
Retries for mttx call
Default 100
--default-sub-address
The default sub address assumed (e.g. for X in CLI and dialled numbers as above) when none added (-X) to queue
Default 9
--no-dial
 -x
 Create queue, but do not dial to send message
--no-wait
Do not wait if a call appears to be in progress
This could have a small window where a mesdsage is queued but not sent, so regular calls to smsq should be done to pick up any missed messages
--concurrent
How many concurrent calls to allow (per queue), default 1
--mr
 -n
 Message reference
--pid
 -p
 Protocol ID
--dcs
Data coding scheme
--udh
Specific hex string of user data header specified (not including the initial length byte)
May be a blank string to indicate header is included in the user data already but user data header indication to be set.
--srr
Status report requested
--rp
Return path requested
--vp
Specify validity period (seconds)
--scts
Specify timestamp (YYYY-MM-DDTHH:MM:SS)
--spool-dir
Spool dir (in which sms and outgoing are found)
Default /var/spool/asterisk

Other arguments starting '-' or '--' are invalid and will cause an error. Any trailing arguments are processed as follows:-

Note that when smsq attempts to make a file in /var/spool/asterisk/outgoing, it checks if there is already a call queued for that queue. It will try several filenames, up to the --concorrent setting. If these files exists, then this means asterisk is already queued to send all messages for that queue, and so asterisk should pick up the message just queued. However, this alone could create a race condition, so if the files exist then smsq will wait up to 3 seconds to confirm it still exists or if the queued messages have been sent already. The --no-wait turns off this behaviour. Basically, this means that if you have a lot of messages to send all at once, asterisk will not make unlimited concurrent calls to the same message centre or device for the same queue. This is because it is generally more efficient to make one call and send all of the messages one after the other.

smsq can be used with no arguments, or with a queue name only, and it will check for any pending messages and cause an outgoing if there are any. It only sets up one outgoing call at a time based on the first queued message it finds. A outgoing call will normally send all queued messages for that queue. One way to use smsq would be to run with no queue name (so any queue) every minute or every few seconds to send pending message. This is not normally necessary unless --no-dial is selected. Note that smsq does only check motx or mttx depending on the options selected, so it would need to be called twice as a general check.

UTF-8 is used to parse command line arguments for user data, and is the default when reading a file. If an invalid UTF-8 sequence is found, it is treated as UCS-1 data (i.e, as is).

The --process option causes smsq to scan the specified queue (default is mtrx) for messages (matching the queue specified, or any if queue not specified) and run a command and delete the file. The command is run with a number of environment variables set as follows. Note that these are unset if not needed and not just taken from the calling environment. This allows simple processing of incoming messages

$queue
 Set if a queue specified
$?srr
 srr is set (to blank) if srr defined and has value 1.
$?rp
 rp is set (to blank) if rp defined and has value 1.
$ud
 User data, UTF-8 encoding, including any control characters, but with nulls stripped out
Useful for the content of emails, for example, as it includes any newlines, etc.
$ude
 User data, escaped UTF-8, including all characters, but control characters \n, \r, \t, \f, \xxx and \ is escaped as \\
Useful fGuaranteed one line printable text, so useful in Subject lines of emails, etc
$ud8
 Hex UCS-1 coding of user data (2 hex digits per character)
Present only if all user data is in range U+0000 to U+00FF
$ud16
 Hex UCS-2 coding of user data (4 hex digits per chartacter)
other
 Other fields set using their field name, e.g. mr, pid, dcs, etc. udh is a hex byte string

File formats

By default all queues are held in a director /var/spool/asterisk/sms. Within this directory are sub directories mtrx, mttx, morx, motx which hold the received messages and the messages ready to send. Also, /var/log/asterisk/sms is a log file of all messages handled.

The file name in each queue directory starts with the queue parameter to SMS which is normally the CLI used for an outgoing message or the called number on an incoming message, and may have -X (X being sub address) appended. If no queue ID is known, then 0 is used by smsq by default. After this is a dot, and then any text. Files are scanned for matching queue ID and a dot at the start. This means temporary files being created can be given a different name not starting with a queue (we recommend a . on the start of the file name for temp files).

Files in these queues are in the form of a simple text file where each line starts with a keyword and an = and then data. udh and ud have options for hex encoding, see below.

UTF-8. The user data (ud) field is treated as being UTF-8 encoded unless the DCS is specified indicating 8 bit formart. If 8 bit format is specified then the user data is sent as is.

The keywords are as follows:-
oa Originating address
The phone number from which the message came
Present on mobile terminated messages and is the CLI for morx messages
da
 Destination Address
The phone number to which the message is sent
Present on mobile originated messages
scts
 The service centre time stamp
Format YYYY-MM-DDTHH:MM:SS
Present on mobile terminated messages
pid
 One byte decimal protocol ID
See GSM specs for more details
Normally 0 or absent
dcs
 One byte decimal data coding scheme
If omitted, a sensible default is used (see below)
See GSM specs for more details
mr
 One byte decimal message reference
Present on mobile originated messages, added by default if absent
srr
 0 or 1 for status report request
Does not work in UK yet, not implemented in app_sms yet
rp
 0 or 1 return path
See GSM specs for details
vp
 Validity period in seconds
Does not work in UK yet
udh
 Hex string of user data header prepended to the SMS contents, excluding initial length byte.
Consistent with ud, this is specified as udh# rather than udh=
If blank, this means that the udhi flag will be set but any user data header must be in the ud field
ud
 User data, may be text, or hex, see below

udh is specified as as udh# followed by hex (2 hex digits per byte). If present, then the user data header indicator bit is set, and the length plus the user data header is added to the start of the user data, with padding if necessary (to septet boundary in 7 bit format).

User data can hold an USC character codes U+0000 to U+FFFF. Any other characters are coded as U+FEFF
ud can be specified as ud= followed by UTF-8 encoded text if it contains no control characters, i.e. only (U+0020 to U+FFFF). Any invalid UTF-8 sequences are treated as is (U+0080-U+00FF).
ud can also be specified as ud# followed by hex (2 hex digits per byte) containing characters U+0000 to U+00FF only.
ud can also be specified as ud## followed by hex (4 hex digits per byte) containing UCS-2 characters.
When written by app_sms (e.g. incoming messages), the file is written with ud= if it can be (no control characters). If it cannot, the a comment line ;ud= is used to show the user data for human readability and ud# or ud## is used.

Delivery reports

The SMS specification allows for delivery reports. These are requested using the srr bit. However, as these do not work in the UK yet they are not fully implemented in this application. If anyone has a telco that does implement these, please let me know. BT in the UK have a non standard way to do this by starting the message with *0#, and so this application may have a UK specific bodge in the near future to handle these.

The main changes that are proposed for delivery report handling are :-