spec.txt.svn-base

来自「cqME :java framework for TCK test.」· SVN-BASE 代码 · 共 416 行

#
# $Id$
#
# Copyright 1996-2007 Sun Microsystems, Inc. All Rights Reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License version
# 2 only, as published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License version 2 for more details (a copy is
# included at /legal/license.txt).
#
# You should have received a copy of the GNU General Public License
# version 2 along with this work; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
# 02110-1301 USA
#
# Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa
# Clara, CA 95054 or visit www.sun.com if you need additional
# information or have any questions.
#

      Communication interfaces specification (draft)
      ==============================================
      
  Testware Components Participating in the Test Run
  -------------------------------------------------

A test harness runs on a PC/Workstation. Its responsibility is to 
package groups of tests into jar files (test bundles) and somehow
make them available to the target device with the implementation 
under test.

A test runner (agent) is prepackaged with each test bundle. Its 
task is to execute tests from this bundle and to report results to 
the test harness.

Additional testware supporting functionality is required to be 
present on the target device. Its task is to repeatedly download 
test bundles from the test harness and execute them.
This functionality is typically implemented through extending JAM.



  Requirements
  ------------
  
1. No dependency on HTTP support for testing (TCK requirement).
   We provide a set of generic interfaces and a sample implementation
   based on HTTP. Users can write their own implementations of these 
   interfaces if they don't support HTTP (but they are still required
   to have some communication channel available).

2. No dependency on JavaTest APIs.
   Interfaces shouldn't include any JavaTest specific references,
   so that it would be possible to use these interfaces outside of TCK
   (that is, on both device side (test runner) and server side (test
   harness) could reside non-TCK software components).



   Solution
   --------
 
The common client-server approach has been chosen. Two new 
communication interfaces are defined - Server and Client. Server is 
a pluggable part of the test harness, Client is a pluggable part of 
the agent. Neither Client nor Server contain any references specific
to the particular test harness or the agent, they operate in terms 
of transferring arrays of bytes.

As a result of this approach, both declared requirements are satisfied.
Pluggability allows for variations of the communication line, and
low-level byte abstraction allows for variations of the test harness
and the agent used.
  


  Java Interfaces
  ---------------

On the device/agent side, the following interface is used:

--------------------------------------------------------------------
public interface Client {

    /**
     * Initialization
     */
    void init(String[] args);

    /**
     * Reads next test.
     * Returns null if all tests from this application have already
     * been executed. In other words, null is an exit signal to the 
     * agent application.
     */
    byte[] getNextTest();

    /**
     * Sends test result.
     */
    void sendTestResult(byte[] res);

}
--------------------------------------------------------------------

This interface allows for any test runner to receive execution requests
and send execution results in a neutral form of a byte array. It is
the responsibility of the particular test harness/agent  pair to
encode/decode the information from bytes to their specific format.


On the PC (Workstation)/test harness side, in addition to the Server,
a concept of the Test Provider is introduced. Test Provider serves as
a server to the Server itself. The Server knows its test provider and
calls its methods in order to pass the data from the client to the 
test provider or vice versa.

Note that in this case the Server has no test related logics inside, 
all it does is just forwarding. It's as lightweight as possible.

The definition of interfaces is as follows:

--------------------------------------------------------------------
public interface Server {

    /**
     * Initialization
     */
    void init(String[] args);

    /**
     * Starting
     */
    void start();

    /**
     * Stopping
     */
    void stop();

    /**
     * Set test provider. There is only one test provider
     * per server. Next call to setTestProvider removes the previous
     * one. Null argument causes removal of current test provider.
     */
    void setTestProvider(TestProvider tp);

    /**
     * Returns current test provider.
     */
    TestProvider getTestProvider();

}
--------------------------------------------------------------------
public interface TestProvider {

    /**
     * Returns main application class for this test provider.
     * The name is the same for all applications generated
     * by this test provider. 
     */
    String getAppMainClass();

    /**
     * Returns the directory in which application jar files
     * are stored.
     */
    String getJarSourceDirectory();

    /**
     * Reads next test.
     * Returns null if all tests from this application have already
     * been executed. In other words, null is an exit signal to the 
     * test runner application.
     */
    byte[] getNextTest();

    /**
     * Sends test result.
     */
    void sendTestResult(byte[] res);

    /**
     * Returns next application to execute. This is a file name 
     * relative to the directory returned by getJarSourceDirectory().
     *
     * May return null if no application is currently available.
     * In this case JAM is expected to repeat request some time later.
     */
    String getNextApp();
    String getNextApp();

}
--------------------------------------------------------------------



  Testware support (JAM)
  ----------------------

The communication between the server and target device's supporting
code is implementation specific. On a high abstraction level,
the device can send the only command to the server - <getNextApp>.
The server, on the other hand, should be able to respond in three 
different ways:
  - <OKAY> + <application descriptor>, if TestProvider.getNextApp()
    returned non-empty string;
  - <RETRY>, possibly with delay time, if TestProvider.getNextApp()
    returned empty string;
  - <DONE>, if TestProvider.getNextApp() returned null.

Let's assume that the target device has JAM that is augmented with
required functionality (so-called test mode). When JAM is started 
into this mode, it performs the following steps:

 1. Contacts the server with the <getNextApp> command.
 2. The server's response may be one of the following:
   2a. <OKAY> + application descriptor.
       JAM downloads and executes the application, and then returns 
       to Step 1.
   2b. <RETRY> command (if no application is currently available).
       JAM waits for some period of time and then returns to Step 1.
       The exact period of wait time is either received from the server
       with this command or it may be a JAM property.
   2c. <DONE> command (if no more applications are expected).
       JAM may either exit the loop or behave similarly to 2b.



  GenericTestProvider Library
  ---------------------------

In order to ease development for this architecture, the GenericTestProvider
Library is offered as a handy tool to implement test providers.

GenericTestProvider is a implementation of the TestProvider interface
that accumulates all of the execution data as a queue of TestBundles.
Each time it receives a request for the next application from the server,
it takes the next bundle from the queue, and returns the name of the 
jar file to the server. Once this file is downloaded and started
to execute, requests for the tests and incoming test results are 
re-addressed to this test bundle. When all of the content of the test 
bundle is executed, GenericTestProvider switches to the next bundle 
in the queue.

The following constructor and method are provided as an interface to 
the test harness:

    public GenericTestProvider(String appMainClass, String jarSourceDir)
    public void dispatchTestBundle(TestBundle bundle)


Accordingly, the TestBundle interface defines a protocol for 
accessing jar file, querying for test requests, reporting test 
results and notification on bundle's live cycle.

--------------------------------------------------------------------
public interface TestBundle {

    /**
     * Returns the name of the jar file
     * for this bundle
     */
    public String getApp();

    /**
     * Reads next test.
     * Returns null if all tests from this bundle have already
     * been executed.
     */
    public byte[] getNextTest();

    /**
     * Reports test result.
     */
    public void passTestResult(byte[] req, byte[] res);

    /**
     * Reports VM exit.
     */
    public void passVMExitResult(byte[] req);
    
    
    /**
     * Notifies this bundle that it is started
     */
    public void starting();

    /**
     * Notifies this bundle that it is restarted
     */
    public void restarting();

    /**
     * Notifies this bundle that it is finished
     */
    public void finishing();
}
--------------------------------------------------------------------

To make this TestBundle abstraction alive, a particular implementation
of it is provided called unsurprisingly GenericTestBundle.
The GenericTestBundle implementation is very basic, it contains only
 - application name (actually, the name of a .jar file);
 - a class to be notified when the results for the tests from this
   bundle are received (TestResultListener);
 - a collection of the test requests.

Accordingly, the test harness should call only

    public void setApp(String app);
    public void setTestResultListener(TestResultListener listener);
    public void addTest(byte[] req);


The TestResultListener interface can be notified both when test
successfully returned some result or when the test never returned
anything (in this case VM exit is assumed).

--------------------------------------------------------------------
public interface TestResultListener {

    void passTestResult(byte[] req, byte[] res);
    
    void passVMExitResult(byte[] req);

}
--------------------------------------------------------------------


I hope the following forwarding chains will help to understand 
this design better:

JAM
|
<getNextApp>  * * * * * * * > Server
                              |
                              GenericTestProvider.getNextApp()
                              | 
                              |-<OKAY> + GenericTestBundle.getApp()
                              |
                              |-<RETRY>
                              |
                              |-<DONE>

Agent
|
Client.getNextTest()  * * * > Server
                              |
                              GenericTestProvider.getNextTest()
                              | 
                              GenericTestBundle.getNextTest()

Agent
|
Client.sendTestResult() * * > Server
                              |
                              GenericTestProvider.sendTestResult()
                              |
                              |-GenericTestBundle.passTestResult()
                              |    |
                              |    TestResultListener.passTestResult()
                              |
                              |-GenericTestBundle.passVMExitResult()
                                   |
                                   TestResultListener.passVMExitResult()



  HTTP Implementation Details
  ---------------------------

Generally, HTTP implementation of the server should work as a 
regular HTTP server. However, there should be some designated
location ("system area"), through which all test requests/responses
should go.
The only requirement is that all communication for this area should
come through uncashed.

Client part is initialized with httpHost, httpPort and systemArea, 
like "http://server:8080/SYSTEM/". Server part is initialized in the 
same way.

Testing related requests go to the server like this:

JAM:

    GET  http://server:8080/SYSTEM/getNextApp      HTTP/1.0

Agent:

    GET  http://server:8080/SYSTEM/getNextTest     HTTP/1.0

    POST http://server:8080/SYSTEM/sendTestResult  HTTP/1.0

Outside of the system area, the server is just a regular HTTP 
server. Only if the requested URL belongs to this area, the
server re-addresses the GET requests and passes results from POST
to the test provider. (Please note that it is absolutely equivalent
to having a servlet or a CGI script at the systemArea location.)

If the test provider returns null for the request either for the 
next test or for the next app, the server replies to the device 
with "404 - Object not found". If the test provider returns empty
string as the next app, the server replies with "503 - Object 
unavailable". In all other cases server returns "200 - OK" and
the provided data.