test_harness.tex

xorp源码hg

%
% $XORP: xorp/docs/test_harness/test_harness.tex,v 1.37 2007/03/15 00:53:48 pavlin Exp $
%

\documentclass[11pt]{article}

%\usepackage[dvips]{changebar}

\usepackage{subfigure}
\usepackage{fullpage}
\usepackage{setspace}
\usepackage{times}
\usepackage{latexsym}
\usepackage{epsfig}
\usepackage{graphicx}
\usepackage{xspace}
\usepackage{color}
\usepackage{amsmath}
\usepackage{rotating}
\usepackage{moreverb}
\usepackage{listings}
\usepackage{alltt}
\usepackage{stmaryrd}
%\usepackage[dvipdf]{graphics}
%\usepackage[dvips]{graphicx}
%\usepackage{xorp}

\definecolor{gray}{rgb}{0.5,0.5,0.5}
\newcommand{\etc}{\emph{etc.}\xspace}
\newcommand{\ie}{\emph{i.e.,}\xspace}
\newcommand{\eg}{\emph{e.g.,}\xspace}
%\newcommand{\comment}[1]{{\color{gray}[\textsf{#1}]}}
%\newcommand{\comment}[1]{}

% Changebar stuff
% \newenvironment{colorcode}{\color{blue}}{}
% \renewcommand{\cbstart}{\begin{colorcode}}
% \renewcommand{\cbend}{\end{colorcode}}

% \pagestyle{empty}

\begin{document}

\title{XORP BGP Test Harness \\
\vspace{1ex}
Version 1.4}
\author{ XORP Project					\\
	 International Computer Science Institute	\\
	 Berkeley, CA 94704, USA			\\
         {\it http://www.xorp.org/}			\\
	 {\it [e-mail]}
}
\date{March 20, 2007}

\maketitle


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% Local definitions
%
\newcommand{\coordinator}{{\em coordinator}\xspace}
\newcommand{\testpeer}{{\em test peer}\xspace}
\newcommand{\testpeers}{{\em test peers}\xspace}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}

This document describes a test harness that was built primarily to
test the XORP BGP implementation. It may be possible to augment the
harness to use it for testing other protocols.

A single BGP process is placed in the harness and tests
are performed on it. The tests can range from testing the decision
process to verifying that a session is dropped when the hold timer expires.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Requirements}

A major requirement was to allow the testing of any BGP process, not
just our own. In the case of our own BGP process it was essential that
regression tests could be run and results verified totally from within
scripts, without the need for any manual configuration. The
XORP BGP regression tests can be run directly from the ``Makefiles''.
\newline

The test harness supports testing at various levels:

\begin{itemize}

  \item Decision process.

  The BGP decision process can be tested by sending update packets to
  BGP, then verifying that the correct update packets are sent by the
  BGP process to the other BGP peers. The actual packets sent by the
  BGP process can be compared to expected packets. 

  Another way that the BGP decision process can be tested is by
  testing the routing tables held at the peers.

  \item Low level testing.

  Testing responses to deliberately corrupted packets.

  \item Load testing. 

  Testing reaction to introducing a large number of routes back
  to back.

  \item Timer testing.

  Correct operation of timers such as the HOLD TIME timer.

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Usage}

\begin{figure}[htbp]
\centerline{\includegraphics[width=0.70\textwidth]{figs/harness}}
\caption{\label{fig:harness}Test harness processes.}
\end{figure}

Figure \ref{fig:harness} shows a BGP process under test that is
connected to two test peers. The test harness has been split into a
number of separate processes. The main reason for using multiple
processes is that a third party BGP processes may not be able to
accept multiple connections from the same host (IP address). The
harness was also simpler to implement by splitting functionality into
separate processes.

The test harness consists of a \coordinator process through which
all interactions with the harness are mediated. There are also one or
more \testpeers . Each \testpeer is capable of forming one BGP
session with the BGP process under test. The \coordinator process
communicates with the \testpeers using XRLs \cite{xorp:xrl}. The
\coordinator process accepts commands via XRLs from test
scripts. Currently our test scripts are written in the shell
programming language, but they could be written in almost any
language. The full set of commands accepted by the \coordinator can be
found in \ref{coord:command}.

Figure \ref{prog:simple} shows an example of a simple program that
might be sent from a test script to a \coordinator. It is assumed that
before the script is sent that all the processes are already running.
One important point to note is that due to the asynchronous nature of
XRLs a command returning does not necessarily mean that it has
completed successfully. A {\em pending} method is available that can
be used to test if all outstanding commands have completed. This
example show the \coordinator being {\em reset}, then its given the
hostname and port number of the BGP process under test. Then the
\coordinator attaches to the two
\testpeers peer1 and peer2, these are the XRL target names by which the
\testpeers are known. The majority of the commands are sent to the
\testpeers themselves. In order to send a command to a \testpeer the
command is preceded by the \testpeer name. In our example each
\testpeer forms a session with the BGP process and then the
{\em assert} command is used to assert that a session is still
established. As noted above the {\em establish} command completing
does not mean that a session was established. Of course attempting to
send a {\em packet} on a session that has not yet been established
will generate an error.

\begin{figure}[htbp]
\small
\begin{verbatim}
reset
target xorp 179
initialise attach peer1
initialise attach peer2
peer1 establish active true AS 1 keepalive true holdtime 0 id 10.10.10.10
peer1 assert established
peer2 establish active true AS 2 keepalive true holdtime 0 id 20.20.20.20
peer2 assert established
\end{verbatim}
\caption{\label{prog:simple}Establish BGP Sessions}
\end{figure}

% An important and overloaded command is the {\em expect} command. Some
% of the commands that are performed by the {\em expect} command should
% really be performed by an {\em assert} command. Apart from the case of
% using {\em expect} to assert that a session has been established, all
% other cases of using {\em expect} are related to verifying that the
% BGP process under test has generated the correct sequence of
% packets.

The {\em expect packet \ldots} command is used to create a queue of
expected packets. Whenever a packet arrives on a peer it is checked
against the queue of expected packets. If there are no expected packets
on the queue then no action is taken. If there is a packet on the
expect queue it is compared against the incoming packet. If the
incoming packet matches,
all is fine and the packet is removed from the queue. If the incoming
packet does not match the packet at the head of the queue, then an
error is flagged and the non matching packet is saved. At the end of a
set of tests the {\em assert queue} command can be used to verify that
queue is at the expected length. If an error has occurred, this is the
point at which the non matching packet and the expected packet are
returned, along with an error status. Figure \ref{prog:assert} is a
example of a code fragment that is waiting for a notify packet on
peer1. A notify packet is added to the queue of expected
packets, then an update packet without an origin is sent to the BGP
process. An update packet without an origin is an error and should
generate the notify packet that is expected. Note that it is
the responsibility of test script to add a delay between sending the
{\em update packet} and the {\em assert}.

\begin{figure}[htbp]
\small
\begin{verbatim}
...
# Update error, missing well known attribute.
peer1 expect packet notify 3 3
# An update packet without an origin
peer1 send packet update aspath 1 nexthop 20.20.20.20 nlri 10.10.10.0/24
# Delay
peer1 assert queue 0
...
\end{verbatim}
\vspace{-0.1in}
\caption{\label{prog:assert}Wait for a notify packet}
\end{figure}

Each {\em peer} has two tries associated with it, a sent and a
received trie. Each update packet that is sent to, or received from,
the BGP process is passed to the appropriate trie. At this time two
types of {\em lookups} can be performed on a trie (Figure
\ref{prog:trie_lookup}). If a {\em lookup} fails then an error is
returned. A test might therefore involve sending many update packets
from different peers and then verifying that the routing tables at the
various peers are correct.  Saving the update packets in the tries
makes it possible to dump the routing tables for post processing.

\begin{figure}[htbp]
\small
\begin{verbatim}
...
peer1 trie recv lookup 212.174.196.0/24
peer1 trie recv lookup 212.174.196.0/24 aspath 1
...
\end{verbatim}
\vspace{-0.1in}
\caption{\label{prog:trie_lookup}Performing lookup in trie}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Commands}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{\label{coord:command}COORD XRLs}

\begin{itemize}

  \item Command("command string")

  Accept commands via XRLs.

  \item Status("peername")

  Returns the status of the named test peer. Can be used to detect if a
  peer is established or not. Plus the number of update messages sent
  or received by the peer. This XRL will not return an error if the
  requested peer does not exist. Can be used in test scripts to wait
  for established, not established transitions. As well as waiting for
  a peering to become quiet because all the state is
  synchronised.

  \item Pending()

  Returns true while there are any uncompleted commands. Can be used
  to poll the coordinator to verify that the previous command has
  completed.

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Commands currently accepted by coord}

\begin{itemize}

  \item {\sf reset}

  Reset all the state in the coordinating process.

  \item {\sf target $<$hostname$>$ $<$port$>$}

  Specify the BGP process under test.

  \item {\sf initialise attach$/$create peername}

  Form an association with a test\_peer. If the second argument is attach then
  it is assumed that the test\_peer is already running. If the second
  argument is create then the test\_peer is started (not currently
  supported).

\end{itemize}

%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Peer specific commands}

\begin{itemize}

  \item connect

  Connect to the BGP target under test.

  \item disconnect

  Disconnect from the BGP target under test.

  \item listen 

  Listen for a connection from the BGP test target.

  \item {\sf establish
  active $<$true/false$>$