results

When a test is run, the results are output in a highly structured heirarchy. The heirarchy reflects the structure of the tests. Thus, there is a separate test result object for each suite, module, class, and test function. The heirachy of these objects is as expected: modules in suites, classes in modules, and tests in classes.

There is also an individual assert result object for each assert/eval function, which is contained within its corresponding test result object. The specific type of assert result object depends on which specific assert/eval function was called.

Assert Result Objects

The following assert result objects form the base of all assert result objects.

class Result(result)

The base object for all assert result objects.

result should be the raw value returned from the comparison function. It will be converted into a three-state variable indicating success: True for pass, False for fail, and None for an unepxected error. This allows an intervening assert result object acces to the return value of a comparison function that doesn’t return a boolean.

class AssertResult(lineno, call, result)

The base for all assert calls. Along with the result of the call, any subclass must provide the line number the call appeared on, as well as the actial call itself.

The following assert result objects are actually raised, and subclass AssertResult. All of them override their calls to str() to output an informative result string.

For AssertTruthResult, AssertEqualsResult, AssertInResult, AssertInstanceResult, and AssertAttribResult, the additional arguments are the parameters that were passed into the corresponding assert/eval function.

Since AssertRaisesResult and ExpectedErrorResult both deal with expected errors, their parameters do not directly map to their data.

class AssertTruthResult(lineno, call, result, value)

The result of a call to assert_true(), assert_false(), eval_true(), or eval_false().

class AssertEqualsResult(lineno, call, result, actual, expected)

The result of a call to assert_equal(), assert_not_equal(), assert_is(), assert_is_not(), assert_none(), assert_not_none(), eval_equal(), eval_not_equal(), eval_is(), eval_is_not(), eval_none(), or eval_not_none().

class AssertInResult(lineno, call, result, item, collection)

The result of a call to assert_in(), assert_not_in(), eval_in(), or eval_not_in().

class AssertInstanceResult(lineno, call, result, obj, cls)

The result of a call to assert_is_instance(), assert_is_not_instance(), eval_is_instance(), or eval_is_not_instance().

class AssertAttribResult(lineno, call, result, obj, attrib_name)

The result of a call to assert_attrib(), assert_not_attrib(), eval_attrib(), or eval_not_attrib().

class AssertRaisesResult(lineno, call, result, expected, func, args, kwargs)

The result of a call to assert_raises() or eval_raises().

result is expected to be a string containing the corresponding traceback, or an empty string to indicate no exception occurred.

class ExpectedErrorResult(result, expected, lineno=None)

This object is not actually the result of a call to an assert function. Rather, it occurs when an error was raised somewhere in the test, and that error was marked as expected in the @test tag. Why is it underneath AssertResult, then? Because it is similar enough that it didn’t make sense to separate it. That being said, I’ve come to the conclusion that it really shouldn’t be here, so it will be moving soon.

Contained within it are the expected error and the lineno it occurred on, if it occurred. If an error was expected but didn’t occur, lineno will be None.

The value for call will always be None since there is no explicit assert call that caused the error. I may be able to detect the line the error actually occurred on, but currently don’t. While this would have some value, it would be tricky to do without making some environment assumptions. Additionally, assert_raises() allows for just this functionality.

The following assert result objects are also actually raised, but inherit directly from Result.

class FailResult(lineno, mess)

Occurs when fail() is called.

The value of result will always be False, since fail() forces the test to fail. lineno is the line that fail occurred on, and mess is the user provided message.

class UnexpectedError(trace)

This is the result when an unexpected error occurs. An unexpected error is an error that is not defined by the @test tag, occurs outside of an assert_raises function, or is not defined for the current assert_raises function.

The value of result will always be None, to indicate an unexpected error. trace contains the traceback of the unexpected error.

Test Result Objects

The following classes form the base for the rest of the test result objects. Note that each of them inherits from list.

class TestResultStruct(name)

The base test result class. All results inherit from this, either directly or indirectly.

class TestResultContainer

The base class for test results objects that are a collection of other test results objects, eg. TestClassResult, TestModuleResult, and TestSuiteResult.

The following test result objects are directly returned when a data object is called.

class TestResult

Contains the assert result objects for each assert that was called in the corresponding test. This is the only test result object which can contain assert result objects, and in fact cannot contain other test result objects.

class TestClassResult

Contains a test result object for each test in this class. Can only contain TestResult objects.

class TestModuleResult

Contains a test result object for each class in this module. Can only contain TestResultClass objects.

class TestSuiteResult

Contains a test result object for each module in this suite. Can only contain TestResultModule objects.

All test result objects (except the base TestResultStruct) contain the following method.

get_status()

Gathers the state of all (test and assert) result objects and returns it. If any unexpected errors have occurred, the test is reported to have terminated with an error. If no unexpected errors occurred but at least one assert failed, the test is reported to have failed. If there were no unexpected errors and no failed asserts, the test passed.

Table Of Contents

Previous topic

data

Next topic

printers

This Page