How I Test

It's been over a year since I've wrote the libellous words "One of the things the PHP community has always been crap about is testing"; an annum on and change is really starting to gain momentum. Personally, I'm now programatically testing pretty much every line of code I ship, in some way or another, and I've learnt a lot over the past year about the best ways to test. Others are starting to really see the value of unit testing.

One of the things I've discovered by talking to other developers is that a lot of people lack direction in what to test and how to make it a part of their process. They get the basics, but they struggle to get into the mindset of someone who writes tests. If you don't get it chances are you're doing it wrong.

Learning the tools is easy. Grabbing PHPUnit off PEAR is a piece of cake, as is setting up a couple of files and running phpunit at your command line. Nowadays, the basic steps of setting up an environment for testing are so few and so effortless, it's child's play. For that reason I don't believe there's much more value in writing another introductory article.

This piece is about the process of testing. It's about test-driven development, how I work when I'm writing code and how writing unit tests gives me smarter, more stable and more refactorable code.

---

Get PHPUnit installed. We're going to write a small PHP class that takes an email request body and parses a bunch of info about it.

Create two new files, email_parser_test.php and email_parser.php. I like to work with these files side by side, so in Sublime Text 2 - a great text editor which you should all be using, incidentally - I'll hit Command + Alt + 2 to open my editor up into two columns. This means I can have my tests on the left and my implementation on the right.

Inside our test file, create a new class and a basic test method:

class Email_parser_test extends PHPUnit_Framework_TestCase
{
    public function test_something()
    {
        $this->assertTrue(true);
    }
}

Jump into Terminal and give it a run:

$ phpunit --colors email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

.

Time: 0 seconds, Memory: 2.25Mb

OK (1 test, 1 assertion)

That --colors flag is going to get annoying, so we'll dump this in a phpunit.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit colors="true"></phpunit>

There's one tiny little bit more of setup we need to do before we can begin writing proper code. We'll need some test data to work with; an email we know the format of and can reproduce every time. Tests are only valuable if they're consistent. We want two emails to work with; and you'll see why in a moment. I'll use an email I got from good friend Alex Bilbie about his excellent CodeIgniter MongoDB Library. Save this in a file email.eml:

Delivered-To: jamie@jamierumbelow.net
Return-Path: <alex.bilbie@gmail.com>
Date: Wed, 2 May 2012 01:06:01 +0100
From: Alex Bilbie <alex.bilbie@gmail.com>
To: jamie@jamierumbelow.net
Message-ID: <586B7E1735704A1CB388F4434B4EAA9B@gmail.com>
Subject: =?utf8?Q?Update_to_my_Mongo_library?=
X-Mailer: sparrow 1.5 (build 1043.1)
Content-Type: text/html
MIME-Version: 1.0

I've updated my CodeIgniter MongoDB library to version 0.5.1. It mostly contains bug fixes and optimisations.

Version 2.0 of the library is almost finished – check it out at <a href="https://github.com/alexbilbie/codeigniter-mongodb-library/tree/v2">https://github.com/alexbilbie/codeigniter-mongodb-library/tree/v2</a>. I'm currently working out how PHP Unit works so that the library can be tested on Travis CI each time a new commit is made.

And another one (email2.eml):

Delivered-To: jamie@jamierumbelow.net
Return-Path: <me@jeremygimbel.net>
Date: Wed, 3 May 2012 09:25:55 +0100
From: Jeremy Gimbel <me@jeremygimbel.net>
To: praise@jamierumbelow.net
Message-ID: <109A8F7538567P7VD675E7990N8NDS7T@gmail.com>
Subject: =?utf8?Q?A_quick_note!?=
Content-Type: text/html
MIME-Version: 1.0

Jamie, I love what you're doing with <a href="https://github.com/jamierumbelow/codeigniter-base-model">MY_Model</a> and <a href="https://github.com/jamierumbelow/codeigniter-base-controller">MY_Controller</a>!

Now we know everything's working a okay and we've got our test data, we'll waste no more time and get straight on with writing our first test.

Actually, that was a lie. I just want to make three points before we begin:

1. Remember that we should always write the smallest amount of code possible to get the test to pass. That'll become clearer shortly.
2. We're writing unit tests, so we're testing the input -> output flow of a unit of code (and any side effects)
3. Your tests should never negatively affect the API. If a parameter is only necessary for the tests, you're doing it wrong.

We'll start off by simply parsing the headers out and grabbing them. This is good place to start because we'll need to parse the headers anyway; it's therefore worth our while to test that core functionality first.

public function test_parses_headers()
{
    $parsed = $this->parser->parse($this->email);
    $this->assertEquals(10, count($parsed['headers']));
}

The test might be simple, and it certainly doesn't test all the functionality we need to, but that is exactly what we want.

When I test, I find it valuable to write the smallest amount of code to test and the smallest possible amount to get the test past. It doesn't matter what the system needs to do in the future, it's just about getting the test to past.

You'll spot that we're accessing $this->parser and $this->email. Let's define them:

public function setUp()
{
    $this->parser = new Email_parser();
    $this->email = file_get_contents('email.eml');
}

Run our tests:

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

PHP Fatal error:  Class 'Email_parser' not found in /Users/jamierumbelow/Sites/email_parser/Email_parser_test.php on line 7

Fatal error: Class 'Email_parser' not found in /Users/jamierumbelow/Sites/email_parser/Email_parser_test.php on line 7

What's the smallest amount of code that can fix this error? In our Emailparser.php_ file:

class Email_parser { }

...and require it at the top of our test file:

require_once 'email_parser.php';

Run again and we get an error about missing parse() method. Define it:

public function parse($email) { }

Run again and we get an assertion error:

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

F

Time: 0 seconds, Memory: 2.75Mb

There was 1 failure:

1) Email_parser_test::test_parses_headers
Failed asserting that 0 matches expected 10.

/Users/jamierumbelow/Sites/email_parser/email_parser_test.php:17

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

What's the smallest amount of code that we can write to make this test pass?

return array( 'headers' => array( 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ) );

You're probably shouting at the screen and telling me I'm an idiot right now, and honestly, I wouldn't blame you. This might seem like a total waste of time. You'll have to bare with me here; you'll soon see why this can become valuable.

Now we've got our test passing, let's add another assertion to it:

public function test_parses_headers()
{
    $parsed = $this->parser->parse($this->email);

    $this->assertEquals(10, count($parsed['headers']));
    $this->assertEquals('jamie@jamierumbelow.net', $parsed['headers']['To']);
}

Got it yet? Nope? Okay, I'll carry on.

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

E

Time: 0 seconds, Memory: 2.50Mb

There was 1 error:

1) Email_parser_test::test_parses_headers
Undefined index: To

/Users/jamierumbelow/Sites/email_parser/email_parser_test.php:18

FAILURES!
Tests: 1, Assertions: 1, Errors: 1.

What's the smallest possible amount of code to fix it? We'll actually cheat a little bit here since we KNOW that once the To index is in the array, it'll ask for the value of jamie@jamierumbelow.net, so we might as well kill two birds with one stone.

return array( 'headers' => array( 'To' => 'jamie@jamierumbelow.net', 2, 3, 4, 5, 6, 7, 8, 9, 10 ) );

Good good. Let's add one more just to ensure we're being thorough:

$this->assertEquals('<586B7E1735704A1CB388F4434B4EAA9B@gmail.com>', $parsed['headers']['Message-ID']);

The failure:

1) Email_parser_test::test_parses_headers
Undefined index: Message-ID

Again, killing two birds with one stone:

return array( 
        'headers' => array( 'To' => 'jamie@jamierumbelow.net', 
                            'Message-ID' => '<586B7E1735704A1CB388F4434B4EAA9B@gmail.com>',
                            3, 4, 5, 6, 7, 8, 9, 10 ) );

So WHAT IS THE POINT OF ALL THIS?

What happens when we copy + paste our assertions and run the same test on email2.eml as well?

Update the assertions so that we're checking for the appropriate number of headers and the appropriate values:

public function test_parses_headers()
{
    $parsed = $this->parser->parse($this->email);

    $this->assertEquals(10, count($parsed['headers']));
    $this->assertEquals('jamie@jamierumbelow.net', $parsed['headers']['To']);
    $this->assertEquals('<586B7E1735704A1CB388F4434B4EAA9B@gmail.com>', $parsed['headers']['Message-ID']);

    $parsed = $this->parser->parse($this->email2);

    $this->assertEquals(9, count($parsed['headers']));
    $this->assertEquals('praise@jamierumbelow.net', $parsed['headers']['To']);
    $this->assertEquals('<109A8F7538567P7VD675E7990N8NDS7T@gmail.com>', $parsed['headers']['Message-ID']);
}

And load our email in our tests' setUp():

$this->email2 = file_get_contents('email2.eml');

Run it:

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

F

Time: 0 seconds, Memory: 2.75Mb

There was 1 failure:

1) Email_parser_test::test_parses_headers
Failed asserting that 10 matches expected 9.

/Users/jamierumbelow/Sites/email_parser/email_parser_test.php:24

FAILURES!
Tests: 1, Assertions: 4, Failures: 1.

Understand the logic yet?

Ahhhhhh. There we go.

Suddenly, we're now in the position where in order to get this test to pass, we're forced to parse the email. In order to get the test passing, we need to implement our test appropriately. Adding in some weird switch as a parameter, although would require less code, detracts from our API (that is to say, it would be pointless in production/implementation code). So the only thing we can really do is actually parse the email.

public function parse($email)
{
    list($headers_list, $body) = explode(PHP_EOL . PHP_EOL, $email, 2);
    $headers_list = explode(PHP_EOL, $headers_list);

    $headers = array();
    foreach ($headers_list as $h)
    {
        list($key, $value) = explode(':', $h, 2);
        $headers[$key] = trim($value);
    }

    return array( 'headers' => $headers );
}

This does two great things. It makes our tests pass:

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

.

Time: 0 seconds, Memory: 2.50Mb

OK (1 test, 6 assertions)

...and it has the fortunate side effect of providing us with the body, parsed and separated out nice and easily!

While it's good that that's already done for us, it's currently untested, so let's fix that posthaste:

public function test_parses_body()
{
    $expected_body_email = "I've updated my CodeIgniter MongoDB library to version 0.5.1. It mostly contains bug fixes and optimisations.\n\nVersion 2.0 of the library is almost finished – check it out at <a href=\"https://github.com/alexbilbie/codeigniter-mongodb-library/tree/v2\">https://github.com/alexbilbie/codeigniter-mongodb-library/tree/v2</a>. I'm currently working out how PHP Unit works so that the library can be tested on Travis CI each time a new commit is made.";
    $expected_body_email2 = 'Jamie, I love what you\'re doing with <a href="https://github.com/jamierumbelow/codeigniter-base-model">MY_Model</a> and <a href="https://github.com/jamierumbelow/codeigniter-base-controller">MY_Controller</a>!';

    $parsed = $this->parser->parse($this->email);
    $this->assertEquals($expected_body_email, $parsed['body']);

    $parsed = $this->parser->parse($this->email2);
    $this->assertEquals($expected_body_email2, $parsed['body']);
}

And making this pass is now just a matter of adding the $body variable to the return line:

return array( 'headers' => $headers, 'body' => $body );

So. We have our headers and we have our body. While cool, it's not very useful. Let's try to extract the subject.

For those of you who know the MIME protocol, you'll know that the subject field is usually encoded in some way. This is conventionally Q-encoding or Base64. PHP's iconv extension provides a great little function for decoding MIME headers: iconv_mime_decode.

Let's write a test:

public function test_subject()
{
    $parsed = $this->parser->parse($this->email);
    $this->assertEquals('Update to my Mongo library', $parsed['subject']);
}

To get it passing:

return array( 'headers' => $headers, 'body' => $body, 'subject' => 'Update to my Mongo library' );

Expand the tests:

$parsed = $this->parser->parse($this->email2);
$this->assertEquals('A quick note!', $parsed['subject']);

And we can implement the function to get the tests to pass:

$subject = iconv_mime_decode($headers['Subject']);

return array( 'headers' => $headers, 'body' => $body, 'subject' => $subject );

And there we have it!

$ phpunit email_parser_test.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /Users/jamierumbelow/Sites/email_parser/phpunit.xml

...

Time: 0 seconds, Memory: 2.50Mb

OK (3 tests, 10 assertions)

It's a simple piece of code and a simple test suite, but hopefully this article has clarified how I test and why I believe this process to be truly valuable.

The CodeIgniter Handbook - Volume 1 - Available!

I am utterly thrilled to announce that my new book, The CodeIgniter Handbook Volume 1, is now available for order!

Volume 1, Who Needs Ruby? is a pragmatic, succinct guide to improving your efficiency and the cleanness of your code. Attractive to amateurs and professionals alike, The CodeIgniter Handbook isn't your usual programming book. It's short, useful and easy to read, and covers how to improve the quality of your code and the speed of your development time.

It's available to buy as a print book or an eBook. It's also much cheaper than most programming books, at only £12 (roughly $18) for the print book and £6 ($10) for the eBook.

I've had some great feedback already -- everyone appears to be enjoying it! Thanks for your support.

Purchase the CodeIgniter Handbook today!

CodeIgniter View Presenters

Whenever building applications with CodeIgniter, I always try my best to follow the Model-View-Controller design pattern as much as possible. MVC tells us that views are the presentational filter of the model's data. Over the course of an application's development, it can be very common for views to grow and accumulate cruft. I find that presenters can be a really sleek way of hiding presentational logic away in a class.

Let's look at a simple view that displays some information about a user's bank account:

<div id="account">
    <h1><?= $this->bank->get($account->bank_id)->name ?> - <?= $account->title ?></h1>

    <p class="information">
        <strong>Name:</strong> <?php if ($account->name): ?><?= $account->name ?><?php else: ?>N/A<?php endif; ?><br />
        <strong>Number:</strong> <?php if ($account->number): ?><?= $account->number ?><?php else: ?>N/A<?php endif; ?><br />
        <strong>Sort Code:</strong> <?php if ($account->sort_code): ?><?= substr($account->sort_code, 0, 2) . "-" . substr($account->sort_code, 2, 2) . "-" . substr($account->sort_code, 4, 2) ?><?php else: ?>N/A<?php endif; ?>
    </p>

    <p class="balances">
        <strong>Total Balance:</strong> <?php if ($account->total_balance): ?><?= "&pound;" . number_format($account->total_balance) ?><?php else: ?>N/A<?php endif; ?>
        <strong>Available Balance:</strong> <?php if ($account->available_balance): ?><?= "&pound;" . number_format($account->available_balance) ?><?php else: ?>N/A<?php endif; ?>
    </p>

    <p class="statements">
        <?php if ($this->statements->get_by('account_id', $account->id)): ?>
            <?= anchor('/statements/' . $account->id, 'View Statements') ?>
        <?php else: ?>
            Statements Not Currently Available
        <?php endif; ?>
    </p>
</div>

This is a rather typical view; it's displaying bits of content from an object, checking for a value's existence and pulling in bits from other database tables. It's fine, but it's all a bit messy, and we're duplicating a fair bit of code. Ideally, we want our view to look something like this:

<div id="account">
    <h1><?= $account->title() ?></h1>

    <p class="information">
        <strong>Name:</strong> <?= $account->name() ?><br />
        <strong>Number:</strong> <?= $account->number() ?><br />
        <strong>Sort Code:</strong> <?= $account->sort_code() ?>
    </p>

    <p class="balances">
        <strong>Total Balance:</strong> <?= $account->total_balance() ?>
        <strong>Available Balance:</strong> <?= $account->available_balance() ?>
    </p>

    <p class="statements"><?= $account->statements_link() ?></p>
</div>

This clears up our view considerably and removes a bunch of the duplication. It also strips out as much logic as possible from the views, and can make for some very succinct code.

We're going to create an application/presenters directory, and inside there an account_presenter.php file. This file will contain the presenter class for our account object. Let's start by extracting the title:

<?php

class Account_Presenter {

    public function __construct($account) {
        $this->account = $account;
        $this->ci =& get_instance();
    }

    public function title() {
        return $this->ci->bank->get($this->account->bank_id)->name . "-" . $account->title;
    }
}

We make sure that we can access the CodeIgniter superobject in $this->ci and the account object that we're presenting in $this->account. We can then port the logic that was previously in the view into a title() method.

We can now go ahead and tidy up the information section:

public function name() {
    return $this->account->name ?: "N/A";
}

public function number() {
    return $this->account->number ?: "N/A";
}

public function sort_code() {
    if ($sc = $this->account->sort_code) {
        return substr($sc, 0, 2) . "-" . substr($sc, 2, 2) . "-" . substr($sc, 4, 2);
    } else {
        return "N/A";
    }
}

Similarly, we can tidy up the balances section:

public function total_balance() {
    return ($this->account->total_balance) ? "£" . number_format($this->account->total_balance) : "N/A";
}

public function available_balance() {
    return ($this->account->available_balance) ? "£" . number_format($this->account->available_balance) : "N/A";
}

And finally, the statements link:

public function statements_link() {
    if ($this->ci->statements->get_by('account_id', $this->account->id)) {
        return anchor('/statements/' . $this->account->id, 'View Statements');
    } else {
        return "Statements Not Currently Available";
    }
}

Fantastic. Finally, in our controller, we need to pass through an instance of the presenter rather than the account object itself.

public function show($account_id) {
    $account = $this->account->get($account_id);
    $this->data['account'] = new Account_Presenter($account);

    $this->load->view('account/show', $this->data);
}

...and don't forget to load the presenter at the top of your controller:

require_once APPPATH . 'presenters/account_presenter.php';

...and we are done!

As I'm sure you can see, using presenters allows you to tidy up your views in a really object-oriented way. It'd be great to hear your thoughts about presenters and whether or not you plan on using them.

Syntax Sugar #3 - Easily Parsing HTML

The PHP SimpleXML extension makes parsing and using XML documents in your code a piece of cake. Unfortunately, HTML rarely complies as a well-formed XML document.

Using SimpleXML combined with DOMDocument, we can parse a reasonably badly formatted HTML document in very few lines of code. The trick here is using the DOMDocument::$strictErrorChecking variable to ensure that the source is parsed as dodgey HTML.

<?php
// Some HTML string...
$html = file_get_contents("http://codeigniter.com");

// Create a new DOMDocument and set strictErrorChecking to FALSE
$dom = new DOMDocument();
$dom->strictErrorChecking = FALSE;

// Load the HTML into the DOMDocument
$dom->loadHTML($html);

// Load the DOMDocument into SimpleXML... and win!
$obj = simplexml_import_dom($dom);

A radical new shift in data storage; Rumblestorage

Last night I had an epiphany. I was working with one of those old, cruddy databases, and it got me thinking. Why isn't there a better solution? Why am I doomed to use legacy systems such as MySQL and MongoDB for the rest of eternity? So, I began researching alternatives. And when I couldn't find any, I chose to write my own.

I am proud to announce Rumblestorage. From now on, every project of mine will be using Rumblestorage as its data storage medium. I hope that you too can see the light and understand that, frankly, databases just don't cut it any more.

Rumblestorage utilises the native capabilities of your language and stores everything in a flat text file; a simple and elegant solution. It brings your data into memory at class instantiation and saves it on destruction. It has a beautifully usable API. It's clear that in this 21st century world, databases are just far too restrictive. Your storage engine should be built specifically for your programming language; it should be fast, easy to use and above all flexible.

Best of all, Rumblestorage is free and open source, available on GitHub. I'm already in talks with the CodeIgniter Reactor team as to integrate Rumblestorage as a replacement for the old and tired database class.

Access the source code at GitHub, and feel free to fork and make your contribution to the future of data storage!