A Simple Lua Test Engine


Running Shake in the command line

Assuming you have a module like LuaFileSystem installed and you go to its /tests directory and run Shake from there, the output would be:

~/workspace/luafilesystem/tests$ shake
->  test.lua OK!

Tests: 27
Failures: 0
Errors: 0

On the other hand, if you have a test script like the one below that includes two assertions that are supposed to fail (lines are numbered):

 1	items = 10
 2	-- checks the correct case
 3	assert (items == 10, "this should not fail")
 5	items = 20
 6	-- checks an overflow case
 7	assert (items == 10, "wrong number of items")
 9	print("Verifying the total")
10	items = 10
11	total = 30
12	assert (items == total, "wrong total")

Shake would register the failures but would run the whole test script, reporting at the end:

:~/workspace$ shake
----------------    test.lua failed!   ----------------

-- checks an overflow case
   #7 assert (items == 10, "wrong number of items")
   items -> 20

Verifying the total
   #12 assert (items == total, "wrong total")
   items -> 10
   total -> 30

Tests: 3
Failures: 2
Errors: 0

Note how much more informative this is when compared to the default output of running the test script with Lua:

:~/workspace$ lua5.1 test.lua
lua5.1: test.lua:7: wrong number of items
stack traceback:
        [C]: in function 'assert'
        test.lua:7: in main chunk
        [C]: ?

Implementing a simple Shake runner

Here we show how to use the Shake API to implement runners. Note that if you just want to execute tests, the built in runners (command line and CGILua app) may be enough. On the other hand, if you need to show more detailed information about the tests results or if you want to use Shake as part of your application runtime, then you will need to use the Shake API.

The minimal Shake runner would be like:

require "shake"
local run = shake.runner()
print (run:summary())

This would be basically equivalent to the command line Shake runner, and if you want to report more details than the default summary does, you will need to drill down the results using something like:

function ReportModules(run)
   local results = run.results
   for cs, suite in ipairs(results.suites) do
       -- displays information about the suite.title
       if suite.error then
	-- displays information about the suite error
       elseif suite.failed > 0 then
	-- displays information about the suite.failed results
	-- displays information about the suite.passed and suite.failed results

For even more details, you may want to inspect each test result and decide what to show:

function ReportModule(run)
  local results = run.results
  for _, suite in ipairs(results.suites) do
    if suite.error == -1 then
      -- displays information about the error
      for _, context in ipairs(suite.contexts) do
        if next(context.tests) then
          if context.output[1] ~= "" or context.comments then
            -- displays information about the context
            for _, output in ipairs(context.output) do
              if output and output ~= "" then
                -- using context.output
            if context.comments and context.comments ~= "" then
              --  or context.comments
          for _, test in ipairs (context.tests) do
	  local linenumber = test.linenumber or "???"
	  local op = test.op
	   local val2 = tostring(test.val2)
	  -- when there is no comparision operator, assume that this is an assert(x) case
	  if not op then
	    val2 = "True value" -- just to diferentiate from the Lua "true"
	  if not op or op == "==" then
	    op = ""
	  -- displays information about the test result using
	  -- linenumber, test.exp1, op, val2, test.val1 and test.msg

Valid XHTML 1.0!

$Id: examples.html,v 1.5 2007/12/21 22:55:20 carregal Exp $