Shinpuru

Overview

Shinpuru is a small testing framework for PHP. It's designed to be as small and simple as possible while still being useful and comfortable. You can use it to test a single class, "your own little blog"™, multiple websites at once, a REST interface or everything you can think of doing with PHP, HTTP, HTML and XML (and that is a lot).

For those who are not familiar with testing yet: Basically it's all about writing small functions that first do something and then verify that an expected result actually occured. For example you can write a test that fetches the index page of your weblog and then verifies that it contains 10 headlines. Here is how this looks like in Shinpuru:

<?php

require('shinpuru.php');

name('Basic usage test');
option('base_url', 'http://arkanis.de');

test('My website should have 10 articles on the front page', function(){
    get('/');
    assert_select('article', 10);
});

?>

When you now run that test script from the command line you'll get a nice report of the test:

$ php basic_usage_test.php
Running: Basic usage test
.
1 test successful
No tests failed, good job!

Of course there is more to Shinpuru than the small example shows. For writing tests Shinpuru has a compact but handy arsenal:

• Basic assertions: Assertions to verify the result of "usual" PHP code.
• HTTP request functions: Functions for common HTTP methods (get(), post(), put(), delete()) and ability to build customize requests in any way.
• HTTP/HTML/XML assertions: Assertions to verify the HTTP response using simple CSS selectors and XPath.
• Flow control assertions using checkpoints and exceptions.

Writing tests is one thing but managing and running tests can get difficult if you have many tests at hand. Shinpuru also tries to help with this:

• Contexts: You can structure your tests into several contexts (e.g. test for the front page, the archive, the backend, etc.) and you can nest contexts if you want.
• Setup and teardown functions: Every context can contain setup and teardown functions that are run before and after each test of the context.
• Autotest: You can run every Shinpuru test in the background. Every time a file is changed the test will be run automatically and a nice confirmation bubble will notify you if all went well or if a test failed.
• Environments: Run or skip tests according to a command line option, e.g. skip database reset when the test is run on the server in the production environment.

All this of course isn't new. In case of Shinpuru it's all heavily inspired by projects from the Ruby world like Rails and Shoulda.

Getting started

If you are new to the whole testing world the next few lines will try to give you a starting point from where you can venture on by your own. In case you are already familiar with testing (or know Shoulda) you better take a look at the features or the reference.

Installation… sort of

To write your own tests with Shinpuru you just have to download and require it. That's all. One download, one require statement and you can start writing your test code. The catch is that you need PHP 5.3 or newer.

That's it for the "installation". Now we can start writing a small test.

A small test

We will start with the absolute bare minimum for a test case:

<?php
require('shinpuru.php');
test('hello world!');
?>

Save the code in the file bare_minimum.php and place it in the same directory as Shinpuru (shinpuru.php). Of course this test will do nothing useful because we just said that we want to test "hello world!" but didn't say how to do it. But for now lets just run it anyway and see what happens. To do this we just execute the test on the command line:

$ php bare_minimum.php
Running: bare_minimum.php
P
1 test passed
No tests failed, good job!

The output shows that one test "passed" (gave up on doing something useful). This is the standard behavior of empty tests and kind of a reminder that we still have to implement this one. You can use this to quickly write a number of tests out of your head and then implement them one after the other.

For now however we want to create a test that actually tests something. Because the world as a whole is somewhat hard to test with a simple PHP script we just try something else: test if a website is online and responds properly.

<?php
require('shinpuru.php');
test('my website', function(){
    get('http://arkanis.de/');
    assert_response(200);
});
?>

We specify the code to test "my website" within an anonymus function. These are new in PHP 5.3 and are quite useful. If you know JavaScript you should be familiar with this kind of functions as well as their syntax. They are also heavily used in the Ruby language.

In the test we fire an HTTP get request at http://arkanis.de/ which happens to be my website but of course you should insert your own website there. After the request we just check that the HTTP response code is 200 which is the code for "everything is alright". Now lets see what happens if we run the test:

$ php my_website.php
Running: my_website.php
.
1 test successful
No tests failed, good job!

Now we have on successful test! The difference to our "hello world!" test is that the test actually did something and was successful in doing so. Therefore it's listed as successful.

Make it break

If everything works out all the time the world would be a very dull place. Lets just make it break a bit by testing an URL that does not exist:

<?php
require('shinpuru.php');
test('my website', function(){
    get('http://explode.arkanis.de/');
    assert_response(200);
});
?>

My website doesn't have an explode subdomain and therefore this code is doomed to fail. Because everyone likes explosions lets take a look how it explodes (but don't take photos, please):

$ php explode_website.php
Running: explode_website.php
F
1 test failed
  - my website (explode_website.php:4): file_get_contents(): php_network_getaddresses: getaddrinfo failed: Name or service not known

Now that was a nice explosion! Actually get() could not find out who explode.arkanis.de is and therefore failed to contact this server. Our assert_response() check is actually never called because get() already exploded and it's useless to continue the test.

For the curious among you: try to request an URL where you get one of those ugly 404 ("file not found") pages and see what the test does.

More tests

One test is nice but often you want to do a bit more. Therefore Shinpuru and other test frameworks provide ways to manage many tests in a logical way:

<?php

require('shinpuru.php');

context('My websites', function(){
    context('layout', function(){
        test('should have a navigation area with links');
        test('should have a non empty tag cloud');
        test('should show my latest projects');
    });
    
    context('fontpage', function(){
        test('should show the 10 latest posts');
    });
});

context('A post', function(){
    test('can have comments');
    
    context('can be formattet with', function(){
        test('HTML');
        test('Markdown');
        test('smilies');
    });
});

?>

This test suite is only a rough map of what should be there and contains no test code at all. Therefore all tests are marked as "passed" (read: please fill in your test code here). So if you just want to make up your mind about what should be in the test suite the names of the test are enought to start with.

You can use contexts to group related tests together and you can nest them as much as you like. If a test fails the context of the test is always shown so you know what's broken.

What next?

Now you have all basic knowledge for writing tests. From here on it's just a matter of experimentation and throwing in new assertions (functions like assert_response()) here and there. It's a good idea to glance though the list of available assertions over in the reference since these are the functions that actually check something. The more assertions you use in your tests the more they cover. ;)

If you're running Ubuntu Linux I strongly recomment that you look at the autotest feature since it will greatly speed up writing tests.

Download

You can directly download shinpuru.php and save it somewhere. This file is all you need to write and run test suites.

If you want to take a look at the whole project with all it's test suites and the project page you can browse or checkout my public subversion repository:

svn checkout http://svn.arkanis.de/projects/tools/shinpuru/trunk/

This will checkout the entire project and you can run the Shinpuru test suite yourself. Be warned however the project and reference page are a mess right now.

Automatic testing

Usually you write your tests in some kind of text editor, save them, switch to the command line and run them. While this isn't much overhead it can add up.

In the Ruby on Rails world there is a tool called autotest that runs the test automatically as soon as files are modified and saved. Someone later added Ubuntu OSD notifications to the mix and this gave writing tests a good flow.

Shinpuru also provides this feature and it's as easy as running a test with the -a (short for --autotest) option. However PHP can't do this out of the box and therefore you need to install the inotify PHP extension as well as the libnotify-bin package. On Ubuntu these commands will do:

sudo apt-get install php5-dev php-pear
sudo pecl install channel://pecl.php.net/inotify-0.1.4
sudo apt-get install libnotify-bin

Now Shinpuru will monitor the current directory (and all child directories) for changes. As soon as something happens the test will run and you get a nice confirmation message showing what happened. This way you can focus on your source code while the test result is shown shortly each time you save the file.

You can modify the behavior of the autotest mode with the --monitor and --silent command line arguments. With --monitor you can make Shinpuru monitor more directories or files, e.g. if your source file you edit is not in a subdirectory of the test. --silent on the other hand will turn off the confirmation bubbles in case you don't like them.

Feedback

Comments and ideas about Shinpuru are always welcome. Please post a comment on my weblog or send me a mail to stephan.soller@helionweb.de. If there is enough interest a public repository and a forum or something like this can be established.

The MIT License

Copyright (c) 2010 Stephan Soller <stephan.soller@helionweb.de>.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.