Underappreciated Perl Code — Test::More::subtest()

Consider a long test that you can break into logical subsections. You could make a flat output with the standard series of TAP ok messages, but Test::More gives a more sophisticated alternative:

use Test::More tests => 3;
pass("First test");
subtest 'An example subtest' => sub {
    plan tests => 2;
    pass("This is a subtest");
    pass("So is this");
pass("Third test");

This works into the TAP:

ok 1 - First test
    ok 1 - This is a subtest
    ok 2 - So is this
ok 2 - An example subtest
ok 3 - Third test

Notice that the subtest has it’s own test count (“1..2“).

Now, what if we decide that we could speed up this test by fork()ing off and running in parallel, or by using some kind of event interface where we can’t predict what order the callbacks are going to run. This would make the TAP output of each process or event get mixed up with the others.

TAP is actually a defined protocol with lots of features that should see more use. One of the proposals for a new version of the protocol is Test Groups, which looks like this:

ok 1
1..2 2 a block
1..3 2.1 another block
ok 2.1.1
ok 2.1.2
ok 2.1.3
ok 2.1 # end of another block
ok 2.2
ok 2 # end of a block
1..3 3 a third block
ok 3.1
ok 3.2
not ok 3 # end of a third block, planned for 3 but only ran 2 tests

Which isn’t the prettiest solution, I’ll admit, but it does solve the interleaving problem. The parser can take the test numbers in any order and put them back together.

The other proposal along these lines is Test Blocks, which are similar to what Test::More::subtest() does:

TAP version 14
begin 1 Object creation
  ok 1 Object created OK
  ok 2 Object isa Flunge::Twizzler
end 1 Object creation
ok 2 Clone OK
begin 3 Methods
  ok 1 has twizzle method
  ok 2 has burnish method
  ok 3 has spangle method
  not ok 4 has frob method
end 3 Methods
ok 4 Resources released

But this doesn’t solve the interleaving problem.

I had mentioned these in a Lighting Talk at YAPC::NA 2012, and immediately got myself warnocked. So this is me standing up and asking to get this moving again :)

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Tagged , , . Bookmark the permalink.

3 Responses to Underappreciated Perl Code — Test::More::subtest()

  1. Chad Granum says:

    Check out Fennec on metacpan, it lets you define test blocks, and they are run in parallel.

  2. Jakub Narebski says:

    Better example would be what happened if one tests in subtest fails. How it is denoted?

    • Timm Murray says:

      Same thing, just “not ok” rather than “ok”. A subtest with a failure is considered a failure for the whole subtest.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.