Looping PHPUnit for Fun and Profit

In Test Driven Development (TDD): By Example Kent Beck describes the process he uses to create new classes. As part of that process, he creates a list of all the things the class should do so he has a todo list of what’s been done and what hasn’t. This is an example from his book:

• $5 + 10 CHF = $10 if rate is 2:1
• $5 * 2 = $10
• Make “amount” private
• Dollar side-effects?
• Money rounding?

As he completes an item he crosses it off so he can see what he has left to do.

• $5 + 10 CHF = $10 if rate is 2:1
• $5 * 2 = $10
• Make “amount” private
• Dollar side-effects?
• Money rounding?

I’ve been fascinated with this concept after I read about it because of how it allows you to keep track of new things that come up while you’re working without pulling yourself out of your current work. I tried paper and pencil at first but that caused me to create a bunch of extra paper “waste”. To combat this, I looked for ways to create my list digitally within the PHPUnit.

Using PHPUnit

PHPUnit allows for you to mark as test incomplete (it also supports skipped tests but I didn’t think that made sense for this work). Marking a test incomplete is as simple as adding the following to your test:

<?php
$this->markTestIncomplete('This test has not been implemented yet.');

Then when you run your test it will be displayed in the output:

Just having it highlighted yellow is helpful but you can add the –verbose flag and see which tests still need to be done.

Using the example from above we can then write out a test class like the following:

<?php
namespace Managertools\Tests\Blame;

class JunkTest extends \PHPUnit\Framework\TestCase
{
    public function test10UsdPlus10ChfEquals10UsdIfRateIs2To1()
    {
        $this->markTestIncomplete(
            'This test has not been implemented yet.'
        );
    }

    public function test5UsdTimes2Equals10()
    {
        $this->markTestIncomplete(
            'This test has not been implemented yet.'
        );
    }

    public function testMakeAmountPrivate()
    {
        $this->markTestIncomplete(
            'This test has not been implemented yet.'
        );
    }

    public function testDollarSideEffects()
    {
        $this->markTestIncomplete(
            'This test has not been implemented yet.'
        );
    }

    public function testMoneyRounding()
    {
        $this->markTestIncomplete(
            'This test has not been implemented yet.'
        );
    }
}

And when we run the tests:

As we’re working through the tests we can see how we’re progressing:

<?php
namespace Managertools\Tests\Blame;

class JunkTest extends \PHPUnit\Framework\TestCase
{
    // snip
    public function test5UsdTimes2Equals10()
    {
        $this->assertTrue(true);
    }
    // snip
}

Looping

When I’m working on a project, I have my code open on one monitor and my unit tests output on another. I get into a cycle of:

1. Save
2. Tab over to the console
3. Up arrow
4. Hit enter
5. Watch unit tests

This is three too many steps for me so I started running the tests in a loop like this:

while(true); do phpunit tests/path/to/file.php; sleep 1; done

This is an infinite loop that runs your test file then waits a second so you can read the output (1 second works well for me but your experience may vary). The downside to this is that this is a LOT to type and I’m not a fan of extra typing. It’s also hard to run multiple test files. My fix was to create a PHP screen that does the same thing but pulls the files to run from a second file.

<?php
#!/usr/bin/env php

while(true) {
    $tests = require('phpunittests.php');

    foreach ($tests as $test) {
        passthru('./vendor/bin/phpunit -c tests --verbose --colors ' . $test);
    }
    sleep(1);
}

Then you need to create a phpunittests.php file that looks like the following:

<?php
// phpunittests.php
return array(
    'tests/path/to/file.php',
);

The two files do what the command line above does but with two important differences. The first is that we can check the first script into source control which means you don’t need to type it over and over again (I would recommend you add phpunittests.php to your .gitignore file as there tends to be a lot of churn with this file). The second is that we can easily swap out which tests we’re running by changing phpunittests.php file because it’s reloaded every loop and doesn’t require the script to be stopped and restarted.

An added bonus to this setup is that because we’re passing a command line argument from the phpunittests.php file (which you should never do in a production system) we can run specific tests within the class:

<?php
// phpunittests.php
return array(
    '--filter testSingleFunction tests/path/to/file.php',
);

Ways to improve this

There are some small problems with this method. The largest being that because the tests are running in a loop with a delay it’s possible to save while the tests are happening which requires two loops before you can actually see the results. I try to keep my tests fast (<10 ms each) so it’s not a huge problem but it can be annoying for the random tests that do take a while.

There are a couple solutions to this problem you could use newt_wait_for_key to pause the script until you tab back into the terminal and press a key or you can use Inotify to only run the scripts when files change.

The other annoying downside to this is that running multiple test files cause multiple runs of PHPUnit you could solve this problem by creating a custom configuration xml for each run but this solution betrays the cleanliness of the php script.