High-level testing PHP applications with Cucumber

One of the things that the PHP community have always been crap about is testing, and honestly, I've never really been much better at it either. This is partly due to the fact that there really aren't any good testing systems out there for PHP and partly due to the laziness of a lot of developers.

Ruby and Rails have been doing a great job of pushing test support in their communities. One of the best new solutions to have sprung up from the Ruby camp is Cucumber, a plain-text based tool that allows you to write user stories and test applications at a high-level. High-level testing is testing the application at the level that the user will see: ensuring that the appropriate text is displayed in appropriate places, ensuring that a certain change is made when the user visits a certain page, et cetera.

Cucumber can be used to test right down to a unit-test level, where individual blocks of code are tested, but this is somewhat limited to Ruby applications only.

Make sure you have the latest version of Ruby installed, jump into Terminal and install the Cucumber Rubygem. We'll also need the Webrat gem, which will give us browser simulation and a matchers API, the RSpec gem, which will give us our testing framework, and mechanize, which is an underlying framework that we'll use with Webrat.

sudo gem install cucumber
sudo gem install rspec
sudo gem install webrat
sudo gem install mechanize

Once you've installed all those gems, check they're installed and ready to use:

$ cucumber --version
0.10.0
$ spec --version
rspec 1.3.0
$ ruby -e 'require "rubygems"; print require("webrat")'
true
$ ruby -e 'require "rubygems"; print require("mechanize")'
true

Inside your project's directory, we're going to create a new directory called features. This is going to contain all of the code related to our testing. Create two new sub-directories too, features/support and features/step_definitions. The former will contain support files (principally our environment/setup file) and the latter will contain the Webrat code to do the actual testing.

Before we can start writing features, we need to set up our environment so that Webrat knows how to connect to our PHP application. Create a features/support/env.rb file.

require 'rspec/expectations'
require 'webrat'
require 'test/unit/assertions'

World(Test::Unit::Assertions)

Webrat.configure do |config|
  config.mode = :mechanize
end

World do
  session = Webrat::Session.new
  session.extend(Webrat::Methods)
  session.extend(Webrat::Matchers)
  session
end

We're loading RSpec, Webrat and Test::Unit (Ruby's built-in unit tester) and we're then adding the Test::Unit::Assertions module to our Cucumber World (scope). We're configuring Webrat to use Mechanize (which will allow Webrat to simulate a web browser), and then we're making a new Webrat session and exposing it to Cucumber.

For the purposes of demonstration, I've decided to test a remote site instead of a local one, and I thought it would be fun to test the CodeIgniter website. We'll write a very simple feature that tests whether we can login and logout successfully.

Create a new feature file inside features called authentication.feature. The name of this file can be anything, as long as it ends with .feature. A feature contains a bunch of scenarios that document a specific user feature in an application. It's then the job of the step definitions to make this code execute, but a helpful by-product of this is that you've got a bunch of feature definitions that can be easily written and read by non-programmers.

First, let's just describe our feature:

Feature: Authentication
  In order to use the CodeIgniter site
  As a user
  I want to be able to login and logout

This is a standard format for features. It doesn't really do anything, it just gives us a general overview of the specific feature we're writing. Let's add a scenario. Remember, indentation is important here.

  Scenario: Login
    Given I am on the home page
    And I am logged out
    When I click on Login
    Then I should see the login page

So, this scenario details when a user clicks the "Login" link on the http://codeigniter.com site. We're saying "Given I am on the home page", which we'll use to point Webrat to the homepage. We'll also check we're logged out (and click on the logout link to ensure this if we are). When the user clicks on the Login link, we want to see the login page.

Describing this in English might not make a lot of sense yet, but stay with me. It will all become a lot clearer when we write the step definitions. Create a new file in features/step_definitions called authentication_steps.rb. We'll start off by defining the "Given I am on the home page" step.

Given /I am on the home page/ do
  visit "http://codeigniter.com"
end

That was easy! We define the steps similar to the way we write them. We're visiting the homepage when that regex matches on the Given clause. Let's move on to "And I am logged out". This is a bit more tricky. "And", much like "But", is there just for readability. It takes the form of the previous clause, so we'll define it as Given.

Given /I am logged out/ do
  click_link 'Logout' if not contain('<a href="http://codeigniter.com/forums/member/login/">Login</a>')
end

We're clicking the 'Logout' link if the page doesn't contain the login link. This ensures we're logged out. Now it's time to implement the "When" step. We use this step to click on the login link. See where this is going?

When /I click on (.*)/ do |link|
  click_link link
end

I've used a regex here to make this a little more re-usable. Now, we can specify any link in our feature and Cucumber will know what to do with it. Finally, "Then".

Then /I should see the login page/ do
  response_body.should have_xpath(%\//input[@name='username']\)
  response_body.should have_xpath(%\//input[@name='password']\)
  response_body.should contain("Login Required")
end

We're using the response_body method to get the page, then using RSpec's should method to ensure that we've got the two input elements - have_xpath is a Webrat method that checks for the existence of matches on XPath queries - and the message "Login Required".

Now, in the Terminal, run cucumber features, where features is the features/ directory. You should see the feature and scenario outputted to the Terminal, with the Scenario steps in green. Congratulations, you just wrote your first Cucumber scenario!

Let's write another scenario. Replace login and password with your own CodeIgniter username and password.

  Scenario: Login processing
    Given I am on the login page
    When I fill in the username input with **login**
    And I fill in the password input with **password**
    And I click the button Submit
    Then I should see the logged in page

And the step definitions:

Given /I am on the login page/ do
  visit "http://codeigniter.com/forums/member/login/"
end

When /I fill in the (.*) input with (.*)/ do |input, value|
  fill_in input, :with => value
end

When /^I click the button (.*)$/ do |button|
  click_button button
end

Then /^I should see the logged in page$/ do
  response_body.should contain("You are now logged in")
end

I've reduced duplication again in a few places, and made some things regexes just in case we need to re-use them. We can use the fill_in method to fill in an input name with a value. Give that a run and you should get more passes.

Finally, let's write and implement the scenario for logging out.

  Scenario: Logout
    Given I am logged in
    And I am on the home page
    When I click on Logout
    Then I should see the logged out page

We only need to implement two steps here; "Given I am logged in" and "Then I should see the logged out page". The other two we've already written.

Given /I am logged in/ do
  visit "http://codeigniter.com"

  if contain('<a href="http://codeigniter.com/forums/member/login/">Login</a>')
    click_link "Login"
    fill_in 'username', :with => "**username**"
    fill_in 'password', :with => "**password**"
    click_button "Submit"
    response_body.should contain("You are now logged in")
  end
end

Then /I should see the logged out page/ do
  response_body.should contain("You are now logged out")
end

Given I am logged in visits the home page, checks if we're logged out, if we are, clicks on the link, submits the form and checks we're logged in successfully. Then I should see the logged out page goes checks that we see the logged out page. It's all pretty self-explanatory.

I hope I've managed to explain how to write Cucumber features. Cucumber really is a fantastic way to do this higher level testing, and you can use it with any platform and framework, including PHP. Check out the Cucumber site if you're interested in finding out more.