tutorial.sgml

c unit test framework

<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.1//EN">

<book lang="en">
  <bookinfo>
    <title>Check Tutorial</title>
    <author><firstname>Arien</firstname><surname>Malec</surname></author>
    <date>24 May, 2002</date>
    <copyright><year>2002</year><holder>Arien Malec</holder></copyright>
    <legalnotice><para>Copyright (c) 2001-2002 Arien Malec. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled &quot;GNU Free Documentation License&quot;.</para>
    </legalnotice>
  </bookinfo>

  <toc></toc>

  <chapter>
   <title>
Introduction
  </title>
  <para>
Check is a unit testing framework for C. It was inspired by similar frameworks that currently exist for most programming languages; the most famous example being JUnit for Java (<ulink url="http://www.junit.org">www.junit.org</ulink>). There is a list of unit test frameworks for multiple languages at <ulink url="http://www.xprogramming.com/software.htm">www.xprogramming.com/software.htm</ulink>. Unit testing has a long history as part of formal quality assurance methodologies, but has recently been associated with the lightweight methodology called Extreme Programming. In that methodology, the characteristic practice involves interspersing unit test writing with coding (&ldquo;test a little, code a little&rdquo;). While the incremental unit test/code approach is indispensable to Extreme Programming, it is also applicable, and perhaps indispensable, outside of that methodology.
  </para>
  <para>
The incremental test/code approach provides three main benefits to the developer:
  </para>
  <orderedlist>
   <listitem>
   <para>
Because the unit tests use the interface to the unit being tested, they allow the developer to think about how the interface should be designed for usage early in the coding process.
   </para>

  </listitem>
   <listitem>
   <para>
They help the developer think early about aberrant cases, and code accordingly.
   </para>

  </listitem>
   <listitem>
   <para>
By providing a documented level of correctness, they allow the developer to refactor (see <ulink url="http://www.refactoring.com">www.refactoring.com</ulink>) aggressively.
   </para>

  </listitem>

  </orderedlist>
  <para>
That third reason is the one that turns people into unit testing addicts. There is nothing so satisfying as doing a wholesale replacement of an implementation, and having the unit tests reassure you at each step of that change that all is well. It is like the difference between exploring the wilderness with and without a good map and compass: without the proper gear, you are more likely to proceed cautiously and stick to the marked trails; with it, you can take the most direct path to where you want to go.
  </para>
  <para>
Look at the Check homepage for the latest information on Check: <ulink url="http://check.sourceforge.net">http://check.sourceforge.net</ulink>
  </para>
  <para>
The Check project page is at <ulink url="http://sourceforge.net/projects/check/">http://sourceforge.net/projects/check/</ulink>
  </para>

  </chapter>

  <chapter>
   <title>
Unit testing in C
  </title>
  <para>
The approach to unit testing frameworks used for Check originated with Smalltalk, which is a late binding object-oriented language supporting reflection. Writing a framework for C requires solving some special problems that frameworks for Smalltalk, Java or Python don't have to face. In all of those language, the worst that a unit test can do is fail miserably, throwing an exception of some sort. In C, a unit test is just as likely to trash its address space as it is to fail to meet its test requirements, and if the test framework sits in the same address space, goodbye test framework. To solve this problem, Check uses to fork system call to create a new address space in which to run each unit test, and then uses message queues to send information on the testing process back to the test framework. That way, your unit test can do all sorts of nasty things with pointers, and throw a segmentation fault, and the test framework will happily note a unit test error, and chug along.
  </para>
  <para>
The Check framework is also designed to play happily with common development environments for C programming. The author designed Check around Autoconf/Automake (thus the name Check -- make check is the idiom used for testing with Autoconf/Automake), and the test failure messages thrown up by Check use the common idiom of filename:linenumber:message used by gcc and family to report problems in source code. With (X)Emacs, the output of Check allows one to quickly navigate to the location of the unit test that failed; presumably that also works in VI and IDEs.
  </para>
   <section>
    <title>
Other unit testing frameworks for C
   </title>
   <para>
The author knows of the following additional unit testing frameworks for C:
   </para>
   <variablelist>
    <varlistentry>
    <term>
GNU&nbsp;Autounit
</term><listitem><para>Much along the same lines as Check, including forking to run unit tests in a separate address space (in fact, the author of Check stole the idea from GNU Autounit). GNU Autounit uses GLib extensively, which means that linking and such need special options, but this may not be a big problem to you, especially if you are already using GTK or GLib. See <ulink url="http://www.recursism.com/s2004/zp/products/gnu+autounit">http://www.recursism.com/s2004/zp/products/gnu+autounit</ulink>.
    </para>

   </listitem>

   </varlistentry>
    <varlistentry>
    <term>
cUnit
</term><listitem><para>Also uses GLib, but does not fork to protect the address space of unit tests. See <ulink url="http://people.codefactory.se/~spotty/cunit/">http://people.codefactory.se/~spotty/cunit/</ulink>.
    </para>

   </listitem>

   </varlistentry>
    <varlistentry>
    <term>
CUnit
</term><listitem><para>Standard C, with plans for an Win32 GUI implementation. Does not currently fork or otherwise protect the address space of unit tests. In early development. See <ulink url="http://cunit.sourceforge.net">http://cunit.sourceforge.net</ulink>.
    </para>

   </listitem>

   </varlistentry>

   </variablelist>
   <para>
It is the author's considered opinion that forking or otherwise trapping and reporting signals is indispensable for unit testing (but it probably wouldn't be hard to add that to cUnit or CUnit). Try 'em all out: adapt this tutorial to use all of the frameworks above, and use whichever you like. Contribute, spread the word, and make one a standard. Languages such as Java and Python are fortunate to have standard unit test frameworks; it would be desirable that C have one as well.
   </para>

   </section>


  </chapter>

  <chapter>
   <title>
Tutorial: Basic unit testing
  </title>
  <para>
This tutorial will use the JUnit Test Infected article (see <ulink url="http://members.pingnet.ch/gamma/junit.htm">Test Infected</ulink>) as a starting point. We will be creating a library to represent money, allowing conversions between different currency types. The development style will be &ldquo;test a little, code a little&rdquo; with unit test writing preceding coding. This constantly gives us insights into module usage, and also makes sure we are constantly thinking about how to test our code.
  </para>
   <section>
    <title>
How to write a test
   </title>
   <para>
Test writing using Check is very simple. The file in which the checks are defined must include check.h as so:
   </para>
   <programlisting>
#include &lt;check.h&gt;</programlisting>
   <para>
The basic unit test looks as follows:
   </para>
   <programlisting>
START_TEST (test_name)
{
  /* unit test code */
}
END_TEST</programlisting>
   <para>
The START_TEST/END_TEST pair are macros that setup basic structures to permit testing. It is a mistake to leave off the END_TEST marker; doing so produces all sorts of strange errors when the check is compiled. 
   </para>

   </section>

   <section>
    <title>
Setting up the money tests
   </title>
   <para>
Since we are creating a library to handle money, we will first create a header money.h, and a file to contain our unit tests, check_money.c (there is a pun there, but no matter...). To manage everything we'll use Autoconf/Automake for this example. (One could do something similar with ordinary makefiles, but in the author's opinion, it is generally easier to use Autoconf/Automake than bare makefiles, and it provides built-in support for running tests). Here is the Makefile.am:
   </para>
   <programlisting>
TESTS=check_money
noinst_PROGRAMS=check_money
check_money_SOURCES=  money.h money.c check_money.c
check_money_INCLUDES= @CHECK_CFLAGS@
check_money_LDADD= @CHECK_LIBS@</programlisting>
   <para>
Special mumbo jumbo to use in configure.in is:
   </para>
   <programlisting>
AC_INIT()
AM_INIT_AUTOMAKE()
AC_PROG_CC
AM_PATH_CHECK()
AC_OUTPUT(Makefile)</programlisting>
   <para>
This will ensure that things link up properly with Check by defining the appropriate compiler and linker flags as CHECK_CFLAGS and CHECK_LIBS. It also makes sure that we can only compile in an environment that has Check. The money.h header should only contain the standard &num;ifndef MONEY_H stuff, money.c should be empty, and check_money.c should only contain an empty main function. Run this with make -k check, after going through the setups to get autoconf and friends working. If all goes well, make should report that our tests passed. No surprise, because there aren't any tests to fail.
   </para>
   <para>
The AM_PATH_CHECK() macro is defined in the file check.m4 which is installed by Check. If you see warnings from automake or aclocal this most certainly means that you forgot to call aclocal or that aclocal can't find check.m4. AM_PATH_CHECK() has some optional parameters that you might find useful:
   </para>
   <programlisting>
AM_PATH_CHECK([MINIMUM-VERSION,[ACTION-IF-FOUND[,ACTION-IF-NOT-FOUND]]])</programlisting>

   <para>
One way to run the autoconf/automake tools is shown below in a simple
shell script. The variable CHECK_DIR should point to the directory
where check.m4 is installed. If you install it in a standard location,
e.g. /usr/share/aclocal, then you won't need the variable defined.
   </para>
   <programlisting>
#!/bin/sh
if [ -n "$CHECK_DIR" ]; then
    aclocal -I $CHECK_DIR
else
    aclocal
fi
autoconf
autoheader
automake --add-missing</programlisting>
   <para>
This shell script and the above configure.in were run on a Red Hat 9
box with Autoconf version 2.13 and automake (GNU automake) 1.4-p6. As
the saying goes, &quot;Your mileage may vary...&quot;
   </para>
   </section>

   <section>
    <title>
Test a little, code a little
   </title>
   <para>
The Test Infected article starts out with a Money class, and so will we. Of course, we can't do classes with C, but we don't really need to. The Test Infected approach to writing code says that we should write the unit test <emphasis>before</emphasis> we write the code, and in this, we will be even more dogmatic and doctrinaire than the authors of Test Infected (who clearly don't really get this stuff, only being some of the originators of the Patterns approach to software development and OO design).
   </para>
   <para>
Here is our first unit test:
   </para>
   <programlisting>
START_TEST (test_create)
{
 Money *m;
 m = money_create(5, "USD");
 fail_unless(money_amount(m) == 5,
             "Amount not set correctly on creation");
 fail_unless(strcmp(money_currency(m), "USD") == 0,
             "Currency not set correctly on creation");
 money_free(m);
}
END_TEST</programlisting>
   <para>
A unit test should just chug along and complete. If it exits early, or is signaled, it will fail with a generic error message. (Note: it is conceivable that you expect an early exit, or a signal. There is currently nothing in Check to specifically assert that we should expect either -- if that is valuable, it may be worth while adding to Check). If we want to get some information about what failed, we need to use the <function>fail_unless()</function> function. The function (actually a macro) takes a first Boolean argument, and an error message to send if the condition is not true.
  </para>
  <para>
If the Boolean argument is too complicated to elegantly express within <function>fail_unless()</function>, there is an alternate function <function>fail()</function>, that unconditionally fails. The second test above can be rewritten as follows:
   </para>
   <programlisting>
if (strcmp(money_currency(m), "USD") != 0) {
  fail("Currency not set correctly on creation");
}</programlisting>
   <para>
There is also a <function>fail_if()</function> function, which is the inverse of <function>fail_unless()</function>, the above test then looks like:
   </para>
   <programlisting>
fail_if(strcmp(money_currency(m), "USD") != 0,
        "Currency not set correctly on creation");</programlisting>
   <para>
For your convenience all fail functions also accepts NULL as the msg argument and substitutes a suitable message for you. So you could also write a test as follows:
   </para>