doc: Introduce documentation for osmo-trx-ipc and its IPC interfaceHEADmaster

Related: SYS#6861
Change-Id: Id6863731f9398720030b16efaaf559e05f2444ed

4 files changed, 405 insertions, 0 deletions

diff --git a/doc/manuals/chapters/code-architecture.adoc b/doc/manuals/chapters/code-architecture.adoc
index 1cb2542..5311dbe 100644
--- a/doc/manuals/chapters/code-architecture.adoc
+++ b/doc/manuals/chapters/code-architecture.adoc
@@ -17,6 +17,7 @@ edge[dir=back, arrowtail=empty]
 7[label = "{UHDDevice|...}"]
 8[label = "{LMSDevice|...}"]
 9[label = "{USRPDevice|...}"]
+10[label = "{IPCDevice|...}"]
 
 2->3[arrowtail=odiamond]
 3->4[constraint=false]
@@ -25,6 +26,7 @@ edge[dir=back, arrowtail=empty]
 6->7
 6->8
 6->9
+6->10
 }
 ----
 
diff --git a/doc/manuals/chapters/ipc_if.adoc b/doc/manuals/chapters/ipc_if.adoc
new file mode 100644
index 0000000..9994693
--- /dev/null
+++ b/doc/manuals/chapters/ipc_if.adoc
@@ -0,0 +1,301 @@
+[[ipc_if]]
+== osmo-trx-ipc IPC Interface
+
+This interface is the one used by _osmo_trx_ipc_ backend to communicate to a
+third party process in charge of driving the lowest layer device-specific bits
+(from now on the Driver).
+
+It consists of a set of Unix Domain (UD) sockets for the control plane, plus a
+shared memory region for the data plane.
+
+Related code can be found in the
+https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc[Transceiver52M/device/ipc/]
+directory in _osmo-trx.git_.
+
+If you are a potential driver implementator, the
+various primitives and data structures are publicly available in header file
+https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h].
+
+=== Control plane
+
+Control plane protocol is transmitted over Unix Domain (UD) sockets using
+message based primitives. Each primitive has a type identified by an integer,
+and each type of primitive has a number of extra attributes attached to it. The
+IPC interface consists of 2 types of UD sockets:
+
+* _Master_ UD socket: One per osmo-trx-ipc process.
+
+* _Channel_ UD socket: One for each channel managed by osmo-trx-ipc process.
+
+The _Driver_ is in all cases expected to take the server role when creating UD
+sockets, while _osmo-trx-ipc_ takes the client role and connects to sockets
+provided by the driver.
+
+=== Master UD socket
+
+During startup, _osmo-trx-ipc_ will try connecting to the _Driver_ Master UD
+socket located in the path provided by its own (VTY) configuration. As a result,
+it means the _Driver_ process must be running and listening on the Master UD
+socket before _osmo-trx-ipc_ is started, otherwise _osmo-trx-ipc_ will fail and
+exit.
+
+Once connected, _osmo-trx-ipc_ will submit a `GREETING_REQ` message primitive
+announcing the maximum supported protocol version (first version ever is `1`,
+increasing over time).
+
+The _Driver_ shall then answer in `GREETING_CNF` message primitive with its own
+maximum supported version (`<=` version received), providing 0 if none is
+supported.
+
+If _osmo-trx-ipc_ receives back the requested version, then both sides agreed
+on the protocol version to use.
+If _osmo-trx-ipc_ receives back a lower version, it shall decide to continue
+with version negotiation using a lower version, until a supported version or 0
+is received. If finally 0 is received, _osmo-trx-ipc_ will disconnect and exit
+with failure.
+
+Once the version is negotiated (`v1` as of current date), _osmo-trx-ipc_ will
+ask for device information and available characeristics to the _Driver_ using
+the `INFO_REQ` message primitive.
+
+The _Driver_ shall then answer with a `INFO_CNF` message
+containing information, such as:
+
+* String containing device description
+
+* Available reference clocks,
+
+* {rx,tx} I/Q scaling factors
+
+* Maximum number of channels supported
+
+* for each channel:
+
+** List of available {rx,tx} paths/antennas.
+
+** {min,max}{rx,tx} gains
+
+** Nominal transmit power
+
+All the information received from the _Driver_ during `INFO_CNF` will be used by
+_osmo-trx-ipc_ to decide whether it can fullfil the requested configuration from
+the user, and proceed to open the device, or exit with a failure (for instance
+number of channels, referece clock or tx/rx antenna selected by the user cannot
+be fullfilled).
+
+_osmo-trx-ipc_ will then proceed to open the device and do an initial
+configuration using an `OPEN_REQ` message, where it will provide the _Driver_
+with the desired selected configuration (such as number of channels, rx/tx
+paths, clock reference, bandwidth filters, etc.).
+
+The _Driver_ shall then configure the device and send back a `OPEN_CNF` with:
+
+* `return_code` integer attribute set to `0` on success or `!0` on error.
+
+* Name of the Posix Shared Memory region where data plane is going to be
+transmitted.
+
+* One path for each channel, containing the just-created UD socket to manage
+that channel (for instance by taking Master UD socket path and appending
+`_$chan_idx`).
+
+* Path Delay: this is the loopback path delay in samples (= used as a timestamp
+offset internally by _osmo-trx-ipc_), this value contains the analog delay as
+well as the delay introduced by the digital filters in the fpga in the sdr
+devices, and is therefore device type and bandwidth/sample rate dependant. This
+can not be omitted, wrong values will lead to a _osmo-trx-ipc_ that just doesn't
+detect any bursts.
+
+Finally, _osmo-trx-ipc_ will connect to each channel's UD socket (see next
+section).
+
+Upon _osmo-trx-ipc_ closing the UD master socket connection, the _Driver_ shall
+go into _closed_ state: stop all processing and instruct the device to power
+off.
+
+TIP: See
+https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
+for the detailed definition of all the related message primitives and data
+types for this socket.
+
+=== Channel UD Socket
+
+This socket can be used by _osmo-trx-ipc_ to start/stop data plane processing or
+change channel's parameters such as Rx/Tx Frequency, Rx/Tx gains, etc.
+
+A channel can be either in _started_ or _stopped_ state. When a channel is
+created (during `OPEN_REQ` in the Master UD Socket), it's by default in
+_stopped_ state. `START_REQ` and `STOP_REQ` messages control this state, and
+eventual failures can be reported through `START_CNF` and `STOP_CNF` by the
+_Driver_.
+
+The message `START_REQ` instructs the _Driver_ to start processing data in the
+data plane. Similary, `STOP_REQ` instructs the _Driver_ to stop processing data
+in the data plane.
+
+Some parameters are usually changed only when the channel is in stopped mode,
+for instance Rx/Tx Frequency.
+
+TIP: See
+https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
+for the detailed definition of all the related message primitives and data
+types for this socket.
+
+=== Data Plane
+
+Data plane protocol is implemented by means of a ring buffer structure on top of
+Posix Shared Memory (see `man 7 shm_overview`) between _osmo-trx-ipc_ process
+and the _Driver_.
+
+The Posix Shared Memory region is created and its memory structure prepared by
+the _Driver_ and its name shared with _osmo-trx-ipc_ during _OPEN_CNF_ message
+in the Master UD Socket from the Control Plane. Resource allocation for the
+shared memory area and cleanup is up to the ipc server, as is mutex
+initialization for the buffers.
+
+==== Posix Shared Memory structure
+
+[[fig-shm-structure]]
+.General overview of Posix Shared Memory structure
+[graphviz]
+----
+digraph hierarchy {
+node[shape=record,style=filled,fillcolor=gray95]
+edge[dir=back, arrowtail=empty]
+
+SHM[label = "{Posix Shared Memory region|+ num_chans\l+ Channels[]\l}"]
+CHAN0[label = "{Channel 0|...}"]
+CHAN1[label = "{Channel 1|...}"]
+CHANN[label = "{Channel ...|}"]
+STREAM0_UL[label = "{UL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"]
+STREAM0_DL[label = "{DL Stream|+ semaphore\l+ read_next\l+ write_next\l+ buffer_size /* In samples */\l+ num_buffers\l+ sample_buffers[]\l}"]
+STREAM1_UL[label = "{UL Stream|...}"]
+STREAM1_DL[label = "{DL Stream|...}"]
+STREAMN_UL[label = "{UL Stream|...}"]
+STREAMN_DL[label = "{DL Stream|...}"]
+BUF_0DL0[label = "{DL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"]
+BUF_0DLN[label = "{DL Sample Buffer ....|...}"]
+BUF_0UL0[label = "{UL Sample Buffer 0|+ timestamp\l+ buffer_size /* In samples */\l+ samples[] = [16bit I + 16bit Q,...]\l}"]
+BUF_0ULN[label = "{UL Sample Buffer ...|...}"]
+
+SHM->CHAN0
+SHM->CHAN1
+SHM->CHANN
+
+CHAN0->STREAM0_DL
+CHAN0->STREAM0_UL
+STREAM0_DL->BUF_0DL0
+STREAM0_DL->BUF_0DLN
+STREAM0_UL->BUF_0UL0
+STREAM0_UL->BUF_0ULN
+
+CHAN1->STREAM1_UL
+CHAN1->STREAM1_DL
+
+CHANN->STREAMN_UL
+CHANN->STREAMN_DL
+}
+----
+
+The Posix Shared Memory region contains an array of _Channels_.
+
+Each _Channel_ contains 2 Streams:
+
+* Downlink _Stream_
+
+* Uplink _Stream_
+
+Each _Stream_ handles a ring buffer, which is implemented as:
+
+* An array of pointers to _Sample Buffer_ structures.
+
+* Variables containing the number of buffers in the array, as well as the
+maximum size in samples for each Sample Buffer.
+
+* Variables containing `next_read` and `next_write` _Sample Buffer_ (its index
+in the array of pointers).
+
+* Unnamed Posix semaphores to do the required locking while using the ring
+buffer.
+
+Each _Sample Buffer_ contains:
+
+* A `timestamp` variable, containing the position in the stream of the first
+sample in the buffer
+
+* A `data_len` variable, containing the amount of samples available to process
+in the buffer
+
+* An array of samples of size specified by the stream struct it is part of.
+
+==== Posix Shared Memory format
+
+The Posix Shared memory region shall be formatted applying the following
+considerations:
+
+* All pointers in the memory region are encoded as offsets from the start
+address of the region itself, to allow different processes with different
+address spaces to decode them.
+
+* All structs must be force-aligned to 8 bytes
+
+* Number of buffers must be power of 2 (2,4,8,16,...) - 4 appears to be plenty
+
+* IQ samples format: One (complex) sample consists of 16bit i + 16bit q, so the
+buffer size is number of IQ pairs.
+
+* A reasonable per-buffer size (in samples) is 2500, since this happens to be
+the ususal TX (downlink) buffer size used by _osmo-trx-ipc_ with the b210 (rx
+over-the-wire packet size for the b210 is 2040 samples, so the larger value of
+both is convenient).
+
+TIP: See
+https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/shm.h[Transceiver52M/device/ipc/shm.h]
+for the detailed definition of all the objects being part of the Posix Shared
+memory region structure
+
+==== Posix Shared Memory procedures
+
+The queue in the shared memory area is not supposed to be used for actual
+buffering of data, only for exchange, so the general expectation is that it is
+mostly empty. The only exception to that might be minor processing delays, and
+during startup.
+
+Care must be taken to ensure that only timed waits for the mutex protecting it
+and the condition variables are used, in order to ensure that no deadlock occurs
+should the other side die/quit.
+
+Thread cancellation should be disabled during reads/writes from/to the queue. In
+general a timeout can be considered a non recoverable error during regular
+processing after startup, at least with the current timeout value of one second.
+
+Should over- or underflows occur a corresponding message should be sent towards
+_osmo-trx-ipc_.
+
+Upon **read** of `N` samples, the reader does something like:
+
+. Acquire the semaphore in the channel's stream object.
+
+. Read `stream->next_read`, if `next_read==next_write`, become blocked in
+another sempahore (unlocking the previous one) until writer signals us, then
+`buff = stream->buffers[next_read]`
+
+. Read `buff->data_len` samples, reset the buffer data (`data_len=0`),
+increment `next_read` and if read samples is `<N`, continue with next buffer
+until `next_read==next_write`, then block again or if timeout elapsed, then  we
+reach conditon buffer underflow and `return len < N`.
+
+. Release the semaphore
+
+Upon **write** of `N` samples, the writer does something like:
+
+. Acquire the semapore in the channel's stream object.
+
+. Write samples to `buff = stream->buffers[next_write]`. If `data_len!=0`,
+signal `buffer_overflow` (increase field in stream object) and probably
+increase next_read`.
+
+. Increase `next_write`.
+
+. If `next_write` was `== next_read`, signal the reader through the other
+semaphore that it can continue reading.
\ No newline at end of file
diff --git a/doc/manuals/chapters/trx-backends.adoc b/doc/manuals/chapters/trx-backends.adoc
index 78bc45d..021c6f4 100644
--- a/doc/manuals/chapters/trx-backends.adoc
+++ b/doc/manuals/chapters/trx-backends.adoc
@@ -71,3 +71,103 @@ with a memory buffer. In this mode, data written to the USRP is actually stored
 in a buffer, and read commands to the USRP simply pull data from this buffer.
 This was very useful in early testing, and still may be useful in testing basic
 Transceiver and radioInterface functionality.
+
+
+[[backend_ipc]]
+=== `osmo-trx-ipc` Inter Process Communication backend
+
+This OsmoTRX model provides its own Inter Process Communication (IPC) interface
+to drive the radio device driver (from now on the Driver), allowing for third
+party processes to implement the lowest layer device-specific bits without being
+affected by copyleft licenses of OsmoTRX.
+
+For more information on such interface, see section <<ipc_if>>.
+
+[[fig-backend-ipc]]
+.Architecture with _osmo-trx-ipc_ and its IPC _Driver_
+[graphviz]
+----
+digraph G {
+        rankdir=LR;
+        MS0 [label="MS"];
+        MS1 [label="MS"];
+        OsmoTRX [label="osmo-trx-ipc", color=red];
+        BTS;
+
+        subgraph cluster_ipc_driver {
+                label = "IPC Driver";
+                color=red;
+                RE [label = "Radio Equipment"];
+                REC [label="Radio Equipment Controller"];
+                RE->REC;
+        }
+
+        REC->OsmoTRX [label="IPC Interface", color=red];
+
+        MS0->RE [label="Um"];
+        MS1->RE [label="Um"];
+        OsmoTRX->BTS [label="bursts over UDP"];
+
+}
+----
+
+A sample config file for this OsmoTRX model can be found in _osmo-trx.git_ https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/doc/examples/osmo-trx-ipc/osmo-trx-ipc.cfg[doc/examples/osmo-trx-ipc/osmo-trx-ipc.cfg]
+
+In the config file, the following VTY command can be used to set up the IPC UD Master Socket _osmo-trx-ipc_ will connect to at startup:
+
+.Example: _osmo-trx-ipc_ will connect to UD Master Socket /tmp/ipc_sock0 upon startup
+----
+dev-args ipc_msock=/tmp/ipc_sock0
+----
+
+==== ipc-device-test
+
+When built with `--with-ipc --with-uhd` configure options, _osmo-trx.git_ will
+build the test program called _ipc-driver-test_. This program implements the
+_Driver_ side of the osmo-trx-ipc interface (see <<ipc_if>> for more
+information) on one side, and also interacts internally with UHD (eg B210 as
+when using osmo-trx-uhd).
+
+You can use this small program as a reference to:
+
+* Test and experiment with _osmo-trx-ipc_.
+
+* Write your own IPC _Driver_ connecting to osmo-trx-ipc.
+
+[[fig-backend-ipc-device-test]]
+.Architecture with _osmo-trx-ipc_ and ipc-device-test as IPC _Driver_
+[graphviz]
+----
+digraph G {
+        rankdir=LR;
+        MS0 [label="MS"];
+        MS1 [label="MS"];
+        SDR;
+        ipc_device_test[label = "ipc-device-test", color=red];
+        OsmoTRX [label="osmo-trx-ipc", color=red];
+        BTS;
+
+        MS0->SDR [label="Um"];
+        MS1->SDR [label="Um"];
+        SDR->ipc_device_test [label="UHD"];
+        ipc_device_test->OsmoTRX [label="IPC Interface", color=red];
+        OsmoTRX->BTS [label="bursts over UDP"];
+}
+----
+
+The code for this app is found here:
+
+* https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/ipc-driver-test.h[Transceiver52M/device/ipc/ipc-driver-test.h]
+
+* https://gitea.osmocom.org/cellular-infrastructure/osmo-trx/src/branch/master/Transceiver52M/device/ipc/ipc-driver-test.c[Transceiver52M/device/ipc/ipc-driver-test.c]
+
+Those files use the server-side (_Driver_ side) code to operate the Posix Shared
+Memory region implemented in files `shm.c`, `shm.h`, `ipc_shm.c` and `ipc_shm.h`
+in the same directory.
+
+Most of the code in that same directory is deliverately released under a BSD
+license (unlike most of _osmo-trx.git_), allowing third parties to reuse/recycle
+the code on their implemented _Driver_ program no matter it being proprietary or
+under an open license. However, care must be taken with external dependencies,
+as for instance shm.c uses the talloc memory allocator, which is GPL licensed
+and hence cannot be used in a proprietary driver.
\ No newline at end of file
diff --git a/doc/manuals/osmotrx-usermanual.adoc b/doc/manuals/osmotrx-usermanual.adoc
index 2d1caad..b768306 100644
--- a/doc/manuals/osmotrx-usermanual.adoc
+++ b/doc/manuals/osmotrx-usermanual.adoc
@@ -35,6 +35,8 @@ include::./common/chapters/vty_cpu_sched.adoc[]
 
 include::./common/chapters/trx_if.adoc[]
 
+include::{srcdir}/chapters/ipc_if.adoc[]
+
 include::./common/chapters/port_numbers.adoc[]
 
 include::./common/chapters/bibliography.adoc[]