Creating a new test case

If you are new to unit testing it is recommended that you actually try the code out as we go. There is actually not very much to type and you will get a feel for the rhythm of test first programming.

To run the examples as is, you need an empty directory with the folders classes, tests and temp. Unpack the SimpleTest framework into the tests folder and make sure your web server can reach these locations.

A new test case

The quick introductory example featured the unit testing of a simple log class. In this tutorial on SimpleTest I am going to try to tell the whole story of developing this class. This PHP class is small and simple and in the course of this introduction will receive far more attention than it probably would in production. Yet even this tiny class contains some surprisingly difficult design decisions.

Maybe they are too difficult? Rather than trying to design the whole thing up front I'll start with a known requirement, namely we want to write messages to a file. These messages must be appended to the file if it exists. Later we will want priorities and filters and things, but for now we will place the file writing requirement in the forefront of our thoughts. We will think of nothing else for fear of getting confused. OK, let's make a test...

<?php
require_once(dirname(__FILE__) . '/simpletest/autorun.php');

class TestOfLogging extends UnitTestCase {
    function testFirstLogMessagesCreatesFileIfNonexistent() {
    }
}
?>

Piece by piece here is what it all means.

The dirname(__FILE__) construct just ensures that the path is taken relative current file.

What is this autorun.php file? This file does the expected work of pulling in the definitions of UnitTestCase. It collects all test classes in the current file and runs them automagically. It does this by setting up an exit handler. More on this later when we look at modifying the output.

The tests themselves are gathered in test case classes. This one, the TestOfLogging class , is typical in extending UnitTestCase. When the test case is invoked by the autorunner it will search for any method within that starts with the name "test". Each of these methods will be executed in the order they are defined in the class.

Our only test method at present is called testFirstLogMessagesCreatesFileIfNonexistent(). There is nothing in it yet.

Now the empty method definition on its own does not do anything. We need to actually place some code inside it. The UnitTestCase class will typically generate test events when run and these events are sent to an observing reporter using methods inherited from UnitTestCase.

Now to add test code...

<?php
require_once(dirname(__FILE__) . '/simpletest/autorun.php');
require_once('../classes/log.php');

class TestOfLogging extends UnitTestCase {
    function testFirstLogMessagesCreatesFileIfNonexistent() {
        @unlink(dirname(__FILE__) . '/../temp/test.log');
        $log = new Log(dirname(__FILE__) . '/../temp/test.log');
        $log->message('Should write this to a file');
        $this->assertTrue(file_exists(dirname(__FILE__) . '/../temp/test.log'));
    }
}
?>

You are probably thinking that that is a lot of test code for just one test and I would agree. Don't worry. This is a fixed cost and from now on we can add tests pretty much as one liners. Even less when using some of the test artifacts that we will use later.

You might also have been thinking that testFirstLogMessagesCreatesFileIfNonexistent is an awfully long method name. Normally that would be true, but here it's a good thing. We will never type this name again, and we save ourselves having to write comments or specifications.

Now comes the first of our decisions. Our test file is called log_test.php (any name is fine) and is in a folder called tests (anywhere is fine). We have called our code file log.php and this is the code we are going to test. I have placed it into a folder called classes, so that means we are building a class, yes?

For this example I am, but the unit tester is not restricted to testing classes. It is just that object oriented code is easier to break down and redesign for testing. It is no accident that the fine grain testing style of unit tests has arisen from the object community.

The test itself is minimal. It first deletes any previous test file that may have been left lying around. Design decisions now come in thick and fast. Our class is called Log and takes the file path in the constructor. We create a log and immediately send a message to it using a method named message(). Sadly, original naming is not a desirable characteristic of a software developer.

The smallest unit of a...er...unit test is the assertion. Here we want to assert that the log file we just sent a message to was indeed created. UnitTestCase::assertTrue() will send a pass event if the condition evaluates to true and a fail event otherwise. We can have a variety of different assertions and even more if we extend our base test cases.

Here is the base list...

assertTrue($x)	Fail unless $x evaluates true
assertFalse($x)	Fail unless $x evaluates false
assertNull($x)	Fail unless $x is not set
assertNotNull($x)	Fail unless $x is set to something
assertIsA($x, $t)	Fail unless $x is the class or type $t
assertNotA($x, $t)	Fail unless $x is not the class or type $t
assertEqual($x, $y)	Fail unless $x == $y is true
assertNotEqual($x, $y)	Fail unless $x == $y is false
assertWithinMargin($x, $y, $margin)	Fail unless $x and $y are separated less than $margin
assertOutsideMargin($x, $y, $margin)	Fail unless $x and $y are sufficiently different
assertIdentical($x, $y)	Fail unless $x === $y for variables, $x == $y for objects of the same type
assertNotIdentical($x, $y)	Fail unless $x === $y is false, or two objects are unequal or different types
assertReference($x, $y)	Fail unless $x and $y are the same variable
assertCopy($x, $y)	Fail unless $x and $y are the different in any way
assertSame($x, $y)	Fail unless $x and $y are the same objects
assertClone($x, $y)	Fail unless $x and $y are identical, but separate objects
assertPattern($p, $x)	Fail unless the regex $p matches $x
assertNoPattern($p, $x)	Fail if the regex $p matches $x
expectError($e)	Triggers a fail if this error does not happen before the end of the test
expectException($e)	Triggers a fail if this exception is not thrown before the end of the test

We are now ready to execute our test script by pointing the browser at it. What happens? It should crash...

The reason is that we have not yet created log.php.

Hang on, that's silly! You aren't going to build a test without creating any of the code you are testing, surely...?

Test Driven Development

Co-inventor of eXtreme Programming, Kent Beck, has come up with another manifesto. The book is called Test driven development or TDD and raises unit testing to a senior position in design. In a nutshell you write a small test first and then only get this passing by writing code. Any code. Just get it working.

You write another test and get that passing. What you will now have is some duplication and generally lousy code. You re-arrange, or "refactor", that code while the tests are passing. This stops you breaking anything. Once the code is as clean as possible you are ready to add more functionality, which you don't do. Instead you add another test for your feature and start the cycle again. Your functionality gets created by trying to pass the tests that define it.

Think of it as an executable specification that you create just in time.

This is a radical approach and one that I feel is incomplete, but it makes for a great way to explain a unit tester! We happen to have a failing, not to say crashing, test right now so let's write some code into log.php...

<?php
class Log {
    
    function __construct($path) {
    }
        
    function message($message) {
    }
}
?>

This is the minimum to avoid a PHP fatal error. We now get the response...

And "TestOfLogging" has failed. SimpleTest uses class names by default to describe the tests, but we can replace the name with our own...

class TestOfLogging extends UnitTestCase {
    function __construct() {
        parent::__construct('Log test');
    }

    function testFirstLogMessagesCreatesFileIfNonexistent() {
        @unlink(dirname(__FILE__) . '/../temp/test.log');
        $log = new Log(dirname(__FILE__) . '/../temp/test.log');
        $log->message('Should write this to a file');
        $this->assertTrue(file_exists(dirname(__FILE__) . '/../temp/test.log'));
    }
}

Giving...

If you want to change the test name, then you'd have to do it by changing the reporter output, which we look at later.

To get the test passing we could just create the file in the Log constructor. This "faking it" technique is very useful for checking that your tests work when the going gets tough. This is especially so if you have had a run of test failures and just want to confirm that you haven't just missed something stupid. We are not going that slow, so...

<?php   
class Log {
    var $path;
        
    function __construct($path) {
        $this->path = $path;
    }
        
    function message($message) {
        $file = fopen($this->path, 'a');
        fwrite($file, $message . "\n");
        fclose($file);
    }
}
?>

It took me no less than four failures to get to the next step. I had not created the temporary directory, I had not made it publicly writeable, I had one typo and I did not check in the new directory to CVS (I found out later). Any one of these could have kept me busy for several hours if they had come to light later, but then that is what testing is for.

With the necessary fixes we get...

Success!

You may not like the rather minimal style of the display. Passes are not shown by default because generally you do not need more information when you actually understand what is going on. If you do not know what is going on then you should write another test.

OK, this is a little strict. If you want to see the passes as well then you can subclass the HtmlReporter class and attach that to the test instead. Even I like the comfort factor sometimes.

Tests as Documentation

There is a subtlety here. We don't want the file created until we actually send a message. Rather than think about this too deeply we will just assert it...

class TestOfLogging extends UnitTestCase {
    function testFirstLogMessagesCreatesFileIfNonexistent() {
        @unlink(dirname(__FILE__) . '/../temp/test.log');
        $log = new Log(dirname(__FILE__) . '/../temp/test.log');
        $this->assertFalse(file_exists(dirname(__FILE__) . '/../temp/test.log'));
        $log->message('Should write this to a file');
        $this->assertTrue(file_exists(dirname(__FILE__) . '/../temp/test.log'));

    }
}

...and find it already works...

Actually I knew it would. I am putting this test in to confirm this partly for peace of mind, but also to document the behaviour. That little extra test line says more in this context than a dozen lines of use case or a whole UML activity diagram. That the test suite acts as a source of documentation is a pleasant side effect of all these tests.

Should we clean up the temporary file at the end of the test? I usually do this once I am finished with a test method and it is working. I don't want to check in code that leaves remnants of test files lying around after a test. I don't do it while I am writing the code, though. I probably should, but sometimes I need to see what is going on and there is that comfort thing again.

In a real life project we usually have more than one test case, so we next have to look at grouping tests into test suites.