architecture.tex.svn-base

在Diameter3588协议的基础上开发的软件

Diameter protocol functions are provided by multiple threads. The thread
handling are based on the ACE thread implementations. The issue of
contention and serialization between components within the
implementation is solved by using protected queues. As with threading,
the queue and locks (which provided protection) are ACE based. Note that
the ACE classes used in the libdiameter library will be mentioned in
this document. However, it is assumed that the reader is familiar with
these ACE classes. Detailed description of these classes is best served
by ACE\cite{ace}.

As shown in Figure~\ref{fig:architecture}, several threads exist within
the Diameter program.  Each of these threads are based on ACE
thread/task classes which not only provides platform abstracted
threading but also a software design pattern that strictly defines the
boundaries of an executing module. Within each thread, a protected
message queue exist that allows threads to pass pre-formed messages to
each other. For the purpose of the Diameter implementation, these queues
will be termed job queues. Further discussion on the use of job queues
are described in the types of threads that exist in the program. These
are as follows:

\subsection{Master Thread\label{sec:mastert}}

A master thread exists to orchestrate the existence of all other threads
in the program. The master thread, also known as the factory, is
responsible for spawning a thread for each active peer connection (both
for connected or accepted communication request) It has the
responsibility of initiating and monitoring local asynchronous
connection request as well as accepting remote connection request. On
success of any of these request, it spawns a thread for the newly
established connection and passes ownership of the connection to the new
thread. An advantage of using ACE for this specific communications
pattern is that the underlying communication transport used is
independent of this pattern. Meaning that the thread spawning behavior
based on successful connections will remain the same whether using TCP
or SCTP as an underlying transport.

The factory is also responsible for maintaining the existence of a
thread pool. This thread pool is used as a load balanced execution
context for session management. Beyond being created and destroyed the
factory, the threads in the pool is fairly independent of the
factory. Details about the thread pool and its role is discussed in the
succeeding topics Section~\ref{sec:architecture}. In addition, the
factory also provides global timer services for all internal timing
needs.  All timer requests are callback based and this global timer runs
within the context of the master thread.

\subsection{Peer Connection Threads\label{sec:peert}}

As mentioned in Section~\ref{sec:mastert}, the factory spawns a thread
to handle each successful peer connection. The responsibility of a peer
thread is to adhere to the peer state machine as stated in the Diameter
draft \cite{basep}, to send and receive Diameter messages destined for
other peers (forwarding) or local processing (session management), to
maintaining the peer table and other functions necessary for
establishing Diameter peer connection.

In libdiameter, a singleton class (AAAPeerService) exist to service the
initialization and termination requirements for transport management. In
this singleton class, the peer table is examined to see if a connections
to locally configure peers be attempted on startup. If so requests are
made to the master thread to initiate asynchronous connection to the
peers. These pending connections are monitored by the master thread for
success or failure. On success, a peer thread is spawned and ownership
of the newly established connection is passed to the thread. In
addition, the singleton contains globally accessible protected databases
that each peer thread requires, i.e. the routing table and the peer
table.

For incoming connections, the master thread is constantly monitoring the
well known Diameter port for connection request. Once, a connection
request is received, a peer thread is spawned. As with connection
attempts, the ownership of the established connection is passed
on. Regardless of whether, connecting or accepting, the peer threads
design pattern is the same. An established connection needs to be
serviced within the thread. The Diameter peer state machine, which is
independent of the design pattern, will dictate the behavior of the
executing thread based on whether it connected or accepted the
request. As an example, in the case of accepted request, the state
machine would need to wait for a complete CER to arrive then determine
whether to accept the peer or not. If accepted, the peer thread may add
this new peer as a dynamic entry into the peer table. As for connecting
peers, it would need to send the CER on start of the state machine.

Since ownership of the connection is passed on to the peer thread, it is
it's responsibility to formally disconnect or handle disconnection
events from the remote peer. These events are common and thus part of
the design pattern. The ACE design pattern used are based on
\begin{verbatim}ACE_Svc_handler\end{verbatim} template class. Also,
since each peer thread has a message queue built in, it has to
constantly monitor the message queue and the established peer connection
and act accordingly once a message from any of those sources appear.

\subsection{Thread Pool\label{sec:pool}}

The thread pool design pattern exists to facilitate the use of load
balancing in session management. All required Diameter protocol
activities beyond the peer task will be performed by a thread in this
pool. The pool is composed of a collection of threads that constantly
monitors a job request queue as shown in
Figure~\ref{fig:architecture}. The job request queue is protected by a
mutex lock that allows only one thread to dequeue an entry at a
time. The entries in these queue contains specific job instructions as
well as data, i.e. parsing a message, look up a session and switch its
state machine, call user subscription etc. The ability to send
instructions, along with any associated data, via this queue gives the
opportunity to assign a pending job to the next available idle
thread. Parallelized load balancing on a fair, first come first serve
basis based on the mutex lock can then be achieved.

The contents of the job queues are generic enough to allow instances of
objects to be enqueued and dequeued. With this ability, a design pattern
based on job instance is used to as a generic way of sending
instructions and data to a thread in the pool for processing. In the
implementation, a job instance is defined as follows:

\begin{figure}[htbp]
\begin{center}
\begin{verbatim}
class AAAJobInstance {
   public:
      AAAJobInstance(void *p) { payload = p; };
      virtual u_int32_t Job() { return (0); };

   protected:
      void* payload;
};
\end{verbatim}
\caption{Generic job instance base class {\bf AAAJobInstance}\label{fig:jobsource}}
\end{center}
\end{figure}

The AAAJobInstance class is used as a base class for purpose specific
derived classes, i.e., message parsing class, timeout event processing
class, etc. As an example, a message parsing class can override
AAAJobInstance and implement the Job() method where actually parsing is
to be done. The raw message itself is reference by payload.  This
derived class can then be injected into a job request queue and the next
idle thread in the pool can dequeue the job instance and invoke it's
Job() method. Hence, all threads in the pool monitors the job request
queue and waits to dequeue the next available job instance so they can
invoke it's Job() method. To continue the parsing example, once the
message has been parsed, the Job() method may also try to determine if
the belongs to an existing session. If it does, it creates a session
message job instance and enqueues it to the job request queue so that
the next available thread can process switch the session state machine
based on the parse message.

The granularity of the task that each Job() method must perform is
restricted to a very specific function. The success of a load balanced
system depends on how well distributed and finely divided each job
is. In libdiameter, the number of threads in the pool is controlled by
local configuration.

\section{Basic Message Processing\label{sec:processing}}

This section describes how incoming and outgoing messages are processed
by the transport and session manager and the message parser so as to
provide more insight to the Open Diameter software architecture.

\subsection{Message Transport and Session Processing\label{sec:msgtransport}}

An incoming message from a remote peer is initially received and
processed by the transport manager within one of the peer threads
servicing the remote peer. Once fully received, the transport manager
can partially parse the header and parts of the message body if needed
to determine where to forward the message. If the message is for another
remote host, the message is enqueued into the job queue of the peer
thread servicing the destination peer. If the message is for local
consumption, the transport manager passes the message to the session
manager by injecting the message into the message parsing job queue, see
Figure~\ref{fig:architecture}. This queue serves as the logical boundary
between the transport manager and session manager on the ingress
path. Once a message is enqueued into the message parsing job queue, a
new thread pool job request is made so that entries in the message
parsing job queue can actually be processed. This new job request is
injected into the thread pools job request queue where the next
available thread can dequeue it and start processesing. On processing,
the message is dequeued from the message parsing job queue and then
completely parsed and passed on directly to the application subscriber
who has specifically registered to receive such a message. After
invoking the application subscription, the job is completed.

Within the subscription, an application may need to query the session
database using the message.  The query will return the appropriate
session object associated with the message. The session object can then
be used to update the session state machine or transmitt a reply to the
received message.  If the parsed message, however, is a base protocol
message used for use only by the Diameter library (ASR,ASA,STR,STA),
then the message subscription is internal, i.e. the library itself
subscribes for those base protocol messages. And hence, the subscription
invocation is also internal and any adjustments that this subscription
makes to the session state machine follows the same process of creating
a new thread pool job request and injecting it into the thread pool job
request queue. This new job is a session job which is again waiting to
be processed by the next available thread in the pool. Once dequeued by
an available thread then updates to the session state machine based on
the associated message can be made.

For outgoing messages generated by an application, the application
invokes a Diameter API transmission class which associates the message