arch.xml

The Kannel Open Source WAP and SMS gateway works as both an SMS gateway, for implementing keyword b

    receives UDP messages from the phones, sends them to wapboxes,
    receives reply messages from the wapboxes, and sends the corresponding
    UDP messages to the phones. Since there can be several wapboxes
    connected to a single bearerbox, the bearerbox needs to route the UDP
    packets so that all packets from the same phone are sent to the same
    wapbox. In practice, the routing problem is simplified so that all
    packets from the same IP number go to the same wapbox, even though
    it is not guaranteed that the IP number hasn't been assigned to a
    new phone. Phones are allocated IP numbers dynamically - when they
    make the data call, they get an IP number, and when they terminate
    the call, the IP number is released and can be allocated to the next
    caller. Thus, it might make sense for the bearerbox to know when a
    WAP session or transaction is finished so that when the new phone
    uses the same IP number, it can be assigned to a different wapbox
    with a lower load. In practice, however, this is unlikely to have
    much benefit, so the added complexity is useless. It will be added,
    of course, if benchmarks later show it to be useful.</para>
    
    <para>The bearerbox uses several internal threads and message
    queues. <xref linkend="fig.bearerbox.arch"> shows their structure.
    There can be multiple UDP sockets open, since the WAP protocol uses
    different UDP port numbers to identify the type of message: whether
    the message is for a session mode transaction or a sessionless
    transaction, whether it uses the security layer, etc. The bearerbox
    has a separate thread of type <function>udp_receiver</function>
    for each such UDP socket. That thread reads the UDP packets,
    converts them to message objects used internally within Kannel,
    and puts those into the <literal>incoming_wdp</literal> queue. All
    <function>udp_receiver</function> threads use the same queue.</para>
    
    <para>The <function>wdp_to_wapboxes</function> thread removes
    the messages from the <literal>incoming_wdp</literal> queue and
    routes them to the correct wapbox. Each wapbox is represented
    inside the bearerbox by a separate message queue and two threads:
    <function>boxc_sender</function> and <function>boxc_reader</function>.
    The <function>wdp_to_wapboxes</function> thread thus moves messages
    from the <literal>incoming_wdp</literal> queue to the wapbox specific
    queues. The <function>boxc_sender</function> thread removes the
    messages from the wapbox specific queue and sends them to the wapbox
    proper via a network connection.</para>
    
    <para>The message traffic in the other direction is a mirror
    image.  The <function>boxc_reader</function> thread reads
    messages from the wapbox, via the network connection, and
    puts them in the <literal>outgoing_wdp</literal> queue. The
    <function>wdp_router</function> thread removes the messages
    from that queue and puts them in the queue specific for the
    socket, so that they will be sent from the correct port. The
    <function>udp_sender</function> thread then removes messages from
    the socket specific queue, converts them into UDP packets, and sends
    those to the phones.</para>

    <para>Multiple queues within the bearerbox were used because
    routing can change at any time. If a wapbox disappears, all messages
    waiting to be sent to it should be sent to other wapboxes instead,
    so that they can be dealt with in a useful manner.</para>

    <para>The bearerbox tries to balance the load between different
    wapboxes. This is implemented using heartbeat messages sent by the
    wapboxes to the bearerbox. Each wapbox computes its own load factor.
    In the current implementation, this is the number of open and pending
    HTTP transactions it has. Periodically, the wapboxes send a heartbeat
    message containing their load factor to the bearerbox. The bearerbox
    remembers the latest load factor it got from each wapbox and assigns
    each new phone to the wapbox with the lowest load factor. This 
    keeps the wapboxes balanced approximately evenly in the long run.
    To make sure a sudden surge of new phones won't all be assigned to
    the same wapbox, the bearerbox increments the load factor it stores
    for each wapbox when it assigns a new phone to it.</para>

	<figure id="fig.bearerbox.arch">
	<title>Bearerbox architecture</title>
	<graphic fileref="bearerbox-arch#FIGTYPE#"></graphic>
	</figure>

</sect1>

<sect1>
<title>The WAP Box</title>
  
	<highlights>
	<para>This section explains how the WAP box works. It lists the
	features of WTP, WSP and WAP Push that are implemented. It covers 
        internal architecture: data structures (state machines, events), 
        thread structure, code modules, inter-module and inter-state machine
	communication, and probably explains the preprocessor trick used
	to ease the implementation immensely.</para>
	</highlights>
	
	<para>When pulling, wapbox reads messages from the bearerbox, 
        maintains internal state for each client and makes HTTP requests on 
        behalf of clients. It responds to messages according to WAP specs.
	The basic duties are pretty simple, but things become somewhat more 
        complex when need to deal with large loads.</para>
	
	<para>Only WTP and WSP are implemented. WTLS exists as a patch,
	but is not covered by this thesis. Only the UDP bearer for WDP
	is supported at the moment, which means that the Wireless Control
	Message Protocol (WCMP) is not implemented.</para>

        <para>Push can be confirmed or unconfirmed. Unconfirmed is simple:
        PPG sends the push content to the bearerbox (essentially asks it to
        do a sms push). If confirmed push is session-oriented, the gateway 
        first asks the phone to establish a session with it. PPG maintains
        session and push data for initiators and send confirmations (result
        notifications) to them, if they have asked it. After both kind of 
        pushes, Kannel <emphasis>can</emphasis> work as a pull gateway.
        </para>

        <para>Both unconfirmed and confirmed push are implemented, but only
        unconfirmed one is tested with a real phone. Wapbox does push over
        SMS, but resulting fetch uses an IP bearer.</para>

<sect2>
<title>Thread structure</title>

   	<para>Each WAP protocol stack layer has its own thread. Push OTA
        protocol implementation is divided between OTA layer (requests)
        and application (indications and confirmations) layer. There are two 
        threads corresponding PAP protocol, one communicating with OTA layer 
        and other accepting push requests from a push initiator (Session-
        oriented WSP and confirmed push, these are treated concurrently,
        because, as mentioned before, WAP Push is followed by a pull.):
	
	    <itemizedlist>
	    
	    <listitem><para>Bearerbox communication layer. This layer
	    corresponds to the WDP layer inside the wapbox; network related
            parts of WDP are implemented in the bearerbox. Wapbox segmentates 
            the push content before it sends it to the bearerbox, and selects
            the bearer. This layer is used for both pushing and pulling.
            </para></listitem>
	    
	    <listitem><para>WTP responder layer. This layer exchanges messages
            with the bearerbox communication thread and the session mode WSP
	    thread, and manages WTP responder state machines. Messages coming
            from the phone are routed to this layer. This layer is used for
            pulling.</para></listitem>

            <listitem><para>WTP initiator layer. This is similar to WTP 
            responder, but now the gateway starts the transaction. This layer
            is used for pushing.</para></listitem>
	    
	    <listitem><para>Session mode WSP layer. This layer exchanges 
            messages with the WTP responder, WTP initiator, the application
            layer and OTA layer, and manages the WSP session, method and 
            push state machines. This is used for both push and pull.</para>
            </listitem>
	    
	    <listitem><para>Application layer. This layer exchanges
	    messages with two WSP layers and makes HTTP requests on their 
            behalf. In addition, it implements indications and confirmations 
            of push OTA protocol for confirmed push, sending them to OTA
            layer. So it is used btoh for pushing and pulling.
            </para></listitem>
	    
	    <listitem><para>HTTP client layer. This layer implements the
	    actual HTTP requests. It is independent of WAP protocol stack, 
            and only implements the HTTP protocol. This layer is used for
            pulling, and it belongs to the application layer.</para></listitem>

            <listitem><para>OTA layer. This layer receives messages from
            PAP layer and makes OTA requests to sessionless and session mode 
            WSP layers. It is used for pushing.</para></listitem>

            <listitem><para>PAP layer. This layer accepts HTTP or HTTPS 
            requests from PI and it is used for pushing. If PPG is not 
            configured for HTTPS, there are three threads: <literal>
            ota_read_thread</literal>, <literal>http_read_thread </literal> 
            and <literal>pap_request_thread</literal>.If it is, there are 
            three or four threads: <literal>https_read_thread</literal> is 
            now mandatory, and <literal>http_read_thread</literal> is 
            optional.</para>

            <para><literal>ota_read_thread</literal> communicates with 
            application layer. It sends push confirmations (called result
            notifications) to the push initiator.</para>

            <para><literal>http_read_thread</literal> and <literal>
            https_read_thread</literal>accept pushes from the push initiator. 
            </para>
            <para><literal>pap_request_thread</literal>parses MIME content 
            sent, compiles pap control document, implements required PPG 
            services and converts push content to the tokenized form. It 
            informs push iniatiator, by sending relevant PAP messages to it, 
            errors happened during these processess and communicates with OTA 
            layer.</para></listitem>

            </itemizedlist>
         </para>
        <para>Push application (PAP and OTA layers) layer are in principle 
        independent of WAP protocol stack, but they do use WTP initiator 
        and WSP pushing facitilies. This means that only WSP session 
        facilities are common with push and pull. For usage of threads when
        pulling, see <xref linkend="wapbox-threads"> and when pushing, see
        <xref linkend="pushbox-threads">. Here arrows marked with '1' refer 
        session establishment, ones marked with '2' content pushing and ones 
        marked with '3' push confirmation.
        </para>

	<figure id ="wapbox-threads">
	<title>Wapbox thread structure for pulling</title>
	<graphic fileref="wapbox-threads#FIGTYPE#"></graphic>
	</figure>

        <para>Sessionless WSP and unconfirmed push services use only a part
        of whole stack:</para>

        <itemizedlist>

        <listitem><para>
        Bearerbox communication layer. This layer is always the same.
        </para></listitem>
        <listitem><para>
        Sessionless WSP layer. This layer exchanges messages with the 
        bearerbox communication layer, the application layer and OTA layer. It
        has no state machines as such, so it is very simple. It is completely
        independent of the WTP and session mode WSP layers, despite the name 
        similarity. It can do both push and pull.
        </para></listitem>
        <listitem><para>
        Application layer. In this case, this layer is used only for pull:
        the phone sends no confirmation messages to the server.
        </para></listitem>
        <listitem><para>
        OTA layer. This is similar for confirmed and unconfirmed pushes.
        </para></listitem>
        <listitem><para>
        PAP layer. Now <literal>ota_read_thread</literal> is missing; it is
        used for receiving phone confirmations.
        </para></listitem>

        </itemizedlist>
	
	<para>All communication between internal (ones communicating with
        other Kannel layers) threads is via message queues. These layers 
        (and therefore the threads) send events to each other, no other 
        form of communication is used. Only the event data structures are 
        exposed by each layer: all other data is internal to the layer, 
        and is only manipulated by the single thread that implements that 
        layer. This reduces the need for locking datastructures, which 
        both simplifies the program code and makes execution faster.</para>

        <para> One layer of WAP Push (PAP) forms the external interface to 
        the push initiator. Therefore it includes a HTTP server thread, 
        which has an external originator for its events. In addition, Push
        implementation has an internal layer (OTA). </para>
	
	<para>The events used correspond to the ones used in the
	WAP protocol specification. Events related PAP protocol correspond 
        attributes of XML document.There are minor differences due to 
        implementation details, but it has been a goal to make the
        implementation follow the specification as closely as possible to 
        reduce errors. However, making push over SMS requires adding quite 
        many additional fields, but these are well separated from 
        ones corresponding protocol primitives.</para>
	
	<para>An earlier design for the gateway spawned a new thread for
	each incoming WDP packet, had all data structures available to all
	threads, and had elaborate locking to get things to work. This
	proved too hard to get working in the everyone-hacks-everything
	style of program development adopted by the project because of
	its open source nature. Experiments also showed that it would
	have been too expensive, computation wise, at high levels of
	usage.</para>
	
	<para>The current design has a static thread structure. All threads
	are started when at program startup, and remain running until
	the program stops. There is a potential scalability problem on
	machines with a larger number of processors than the wapbox uses:
	many of the processors will be completely idle. It is, however,
	possible to run multiple wapboxes, so it should be easy to utilize
	them anyway.</para>

	<para>The implementation of each layer is basically the same:
	each has a queue of incoming events, to which other layers
	append events. The layer extracts an event from the queue,
	handles it, and possibly sends other events to other layers.
	The only locking needed is when accessing the event queue for
	each layer. The queue operations are very fast (constant time),
	so the time spent waiting for a lock is short.
	</para>

        <figure id="pushbox-threads">
	<title>Wapbox thread structure for pushing.</title>
	<graphic fileref="pushbox-threads#FIGTYPE#"></graphic>
	</figure>

</sect2>


<sect2>
<title>Implementation of protocol state machines</title>

	<para>The WTP and connection mode WSP layers are implemented in
	terms of <glossterm>state machines</glossterm>, as