instrumentation_testing.html

Android Porting Guide

    <td valign="top">The instrumentation test runner is an instrumentation that runs instrumentation test cases and injects itself into each test case. Instrumentation test cases need to be grouped together with an instrumentation test runner with the appropriate target package.</td>
  </tr>
  <tr>
    <td valign="top"><code>InstrumentationTestSuite</code></td>
    <td valign="top">The instrumentation test suite is a simple extension of the standard JUnit <code>TestSuite</code> that keeps a member <code>Instrumentation</code> variable on hand to inject into each <code>TestCase</code> before running them.  It is used by <code>InstrumentationTestRunner</code>.</td>
  </tr>
</table>
<p> Three additional base classes extend <code>InstrumentationTestCase</code> to allow you to test <code>Activity</code> and <code>Provider</code> classes:</p>
<table>
  <tr>
    <th scope="col">Class</th>
    <th scope="col">Description</th>
  </tr>
  <tr>
    <td valign="top"><code>ActivityTestCase</code></td>
    <td valign="top"><p>This class can be used to write tests for a specific activity.  An activity is launched in its <code>setUp()</code> method and finished with <code>tearDown</code>.  If you write a test case that extends <code>ActivityTestCase</code>, you can write tests that access the activity using <code>getActivity()</code> and assume it has been set up properly.</p></td>
  </tr>
  <tr>
    <td valign="top"><code>ServiceTestCase</code></td>
    <td valign="top">This test case provides a framework in which you can test Service classes in a controlled environment.  It provides basic support for the lifecycle of a Service, and hooks by which you can inject various dependencies and control the environment in which your Service is tested.</td>
  </tr>
  <tr>
    <td valign="top"><code>SingleLaunchActivityTestCase</code></td>
    <td valign="top">This class is similar to <code>ActivityTestCase</code> except that the activity is launched once per class instead of every time the test case calls setup. </td>
  </tr>
  <tr>
    <td valign="top"><code>ProviderTestCase</code></td>
    <td valign="top">This class is similar to <code>ActivityTestCase</code> except that it will setup, tear down, and provide access to the <code>Provider</code> of your choice.</td>
  </tr>
</table>


<a name="androidInstrumentationFrameworkamCommand"></a><h3>Understanding the am Command</h3>

<p>The am command is a command-line interface to the ActivityManager (see <a href="http://code.google.com/android/reference/android/app/ActivityManager.html">http://code.google.com/android/reference/android/app/ActivityManager.html</a> for details). <code>am</code> is used to start and instrument activities using the adb shell command, as shown in the snippet below:</p>
<pre class="prettify">
&gt; adb shell am
usage: am [start|instrument]
       am start [-a &lt;ACTION&gt;] [-d &lt;DATA_URI&gt;] [-t &lt;MIME_TYPE&gt;]
                [-c &lt;CATEGORY&gt; [-c &lt;CATEGORY&gt;] ...]
                [-e &lt;EXTRA_KEY&gt; &lt;EXTRA_VALUE&gt; [-e &lt;EXTRA_KEY&gt; &lt;EXTRA_VALUE&gt; ...]
                [-n &lt;COMPONENT&gt;] [-D] [&lt;URI&gt;]
       am instrument [-e &lt;ARG_NAME&gt; &lt;ARG_VALUE&gt;] [-p &lt;PROF_FILE&gt;]
                [-w] &lt;COMPONENT&gt;
For example, to start the Contacts application you can use
&gt; adb shell am start -n com.google.android.contacts/.ContactsActivity
</pre>


<a name="androidInstrumentationFrameworkPlatform"></a><h2>Platform Test Suites</h2>

<p>This section provides an overview for various unit and functional test cases that can be executed through the instrumentation framework.</p>


<a name="androidTestingPlatformFramework"></a><h3>Framework Tests</h3>

<p>Framework test cases test the Android application framework or specific Android application functionality that requires an Android runtime context.  These tests can be found in <code>//device/tests</code> and <code>//device/apps/AndroidTests</code>.</p>


<a name="androidTestingPlatformCoreLibrary"></a><h3>Core Library</h3>

<p>Core library test cases test the Android library functionality that does not require an Android runtime context.  These tests are split into Android library (android.* package space) tests at <code>//device/java/tests</code> and Java library (java.*, javax.*, etc. packages) tests at <code>//device/dalvik/libcore/.../tests</code>.</p>


<a name="androidInstrumentationFrameworkWritingRunning"></a><h2>Running Tests</h2>

<p>Each instrumentation test case is similar to an Android application with the distinction that it starts another application. For example, have a look in the <code>tests/Contacts</code> directory. </p>
<ul>
  <li> There should be a Makefile and an Android Manifest file. </li>
  <li> Tests are located in <code>tests/Contacts/src/com/google/android/contactstests</code>. </li>
  <li> The Instrumentation Test Runner is located at <code>tests/Contacts/src/com/google/android/contactstests/functional/ContactsInstrumentationTestRunner.java</code>.</li>
</ul>
<p>Suppose you have a makefile with <code>Contactstests</code> as the target. </p>
<ul>
  <li> <code>make Contactstests</code>: Compiles the test cases. </li>
  <li> <code>adb install Contactstests.apk</code>: Installs the apk on the device. </li>
  <li> Use the adb shell <code>am</code> command to run them. </li>
</ul>
<p> To run your tests, use the <code>am instrument</code> command with your <code>InstrumentationTestRunner</code> as its argument. Results are printed as a result of the instrumentation. For example, the following snippet displays the output after running the framework tests with one test failing (note the unusual syntax caused by how instrumentations are run via <code>am</code>):</p>
<pre class="prettify">
$ adb shell am instrument -w com.google.android.frameworktest/.tests.FrameworkInstrumentationTestRunner
INSTRUMENTATION_RESULT: test results:=.......F.......
Time: 6.837
There was 1 failure:
1) testSetUpConditions(com.google.android.frameworktest.tests.focus.RequestFocusTest)junit.framework.AssertionFailedError: requestFocus() should work from onCreate.
        at com.google.android.frameworktest.tests.focus.RequestFocusTest.testSetUpConditions(RequestFocusTest.java:66)
        at java.lang.reflect.Method.invokeNative(Native Method)
        at android.test.InstrumentationTestSuite.runTest(InstrumentationTestSuite.java:73)
        at android.test.InstrumentationTestSuite.runTest(InstrumentationTestSuite.java:73)
        at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:151)
        at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1088)

FAILURES!!!
Tests run: 14,  Failures: 1,  Errors: 0

&lt;RETURN&gt; to continue

INSTRUMENTATION_CODE: -1
$ 
</pre>


<a name="androidInstrumentationTestingRunningAll"></a><h3>All Tests with Default TestRunner behavior</h3>

<p>If no class or package is passed in to run, InstrumentationTestRunner will automatically find and run all tests under the package of the test application (as defined by the <code>android:targetPackage</code> attribute of the instrumentation defined in its manifest file).
</p> 
<pre> 
$ adb shell am instrument -w \
  com.android.samples.tests/android.test.InstrumentationTestRunner
 
INSTRUMENTATION_RESULT: Test results for InstrumentationTestRunner=..........
Time: 2.317
 
OK (10 tests)
 
 
INSTRUMENTATION_CODE: -1
</pre>


<a name="androidTestingTestSinglePakcage"></a><h3>Running all Tests Under Single Package</h3>

<p>If you have many tests under one package, use the <code>-e package &lt;packagename&gt;</code> option to run all tests under that package without having to manually create a test suite.</p> 
<pre> 
$ adb shell am instrument -w \
  -e package com.android.samples.view \
  com.android.samples.tests/android.test.InstrumentationTestRunner
INSTRUMENTATION_RESULT: Test results for InstrumentationTestRunner=........
Time: 1.587
 
OK (8 tests)
</pre>  


<a name="androidTestingSingleTestSuite"></a><h3>Running a Single Test Suite</h3>

<p>If you prefer to explicitly state which tests comprise all of your tests, you can define a test suite and run that directly. By convention, all test packages in your system should have at least one suite called <code>AllTests</code> (see <code>AllTests.java</code>).  To run all of the tests using the <code>AllTests</code> suite for the api demos test app:</p>

<pre> 
$ adb shell am instrument -w \
  -e class com.android.samples.AllTests \
  com.android.samples.tests/android.test.InstrumentationTestRunner
 
INSTRUMENTATION_RESULT: Test results for AllTests=..........
Time: 2.286
 
OK (10 tests)
 
 
INSTRUMENTATION_CODE: -1
</pre> 


<a name="androidInstrumentationTestingRunningSingleTestCase"></a><h3>A Single Test Case</h3>

<pre> 
$ adb shell am instrument -w \
  -e class com.android.samples.view.Focus2ActivityTest \
  com.android.samples.tests/android.test.InstrumentationTestRunner
 
INSTRUMENTATION_RESULT: Test results for Focus2ActivityTest=....
Time: 1.359
 
OK (4 tests)
 
 
INSTRUMENTATION_CODE: -1
</pre> 


<a name="androidInstrumentationTestingRunningSingleTest"></a><h3>A Single Test</h3>

<pre> 
$ adb shell am instrument -w \
  -e class com.android.samples.view.Focus2ActivityTest#testGoingLeftFromRightButtonGoesToCenter \
  com.android.samples.tests/android.test.InstrumentationTestRunner
 
INSTRUMENTATION_RESULT: Test results for Focus2ActivityTest=.
Time: 0.51
 
OK (1 test)
 
 
INSTRUMENTATION_CODE: -1
</pre> 


<a name="androidTestingDebugging"></a><h3>Attaching a debugger to your test</h3>

<p>In order to debug your test code, instruct the controller to stop and wait for the debugger by adding <code>-e debug true</code> to your
command line.  This causes the test runner to stop and wait for the debugger just before calling your <code>setUp()</code> method.  For example,</p> 

<pre> 
$ adb shell am instrument -w \
  -e debug true \
  com.android.samples.tests/android.test.InstrumentationTestRunner
</pre> 


<a name="androidInstrumentationTestingCreating"></a><h2>Writing Tests</h2>

<p>When writing tests, refer to the ApiDemos tests as models (located at <code>//device/samples/ApiDemos</code>). This section provides an overview of the test structure with ApiDemos.</p>


<a name="androidTestingLocationFiles"></a><h3>Location of Files</h3>

<p>Test packages should use the following structure and include <code>Android.mk</code>, <code>AndroidManifest.xml</code>, <code>AllTests.java</code>, and a src directory that mirrors the src directory of the tested application.</p> 
<p>Files are located within a <code>tests</code> directory found in the root directory:</p> 
<pre> 
$ find samples/ApiDemos/tests
samples/ApiDemos/tests
samples/ApiDemos/tests/Android.mk
samples/ApiDemos/tests/AndroidManifest.xml
samples/ApiDemos/tests/src
samples/ApiDemos/tests/src/com
samples/ApiDemos/tests/src/com/google
samples/ApiDemos/tests/src/com/google/android
samples/ApiDemos/tests/src/com/google/android/samples
samples/ApiDemos/tests/src/com/google/android/samples/AllTests.java
samples/ApiDemos/tests/src/com/google/android/samples/ApiDemosTest.java
samples/ApiDemos/tests/src/com/google/android/samples/os
samples/ApiDemos/tests/src/com/google/android/samples/os/MorseCodeConverterTest.java
samples/ApiDemos/tests/src/com/google/android/samples/view
samples/ApiDemos/tests/src/com/google/android/samples/view/Focus2ActivityTest.java
samples/ApiDemos/tests/src/com/google/android/samples/view/Focus2AndroidTest.java
</pre>


<a name="androidTestingContentMakefile"></a><h3>Contents of makefile</h3>

<p>The contents of the makefile are similar to a normal application with the addition of a <code>LOCAL_INSTRUMENTATION_FOR</code> declaration.<p /> 
<pre> 
# Add appropriate copyright banner here
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
 
# We only want this apk build for tests.
LOCAL_MODULE_TAGS := tests