qm.test.result.Result

Package qm :: Package test :: Module result :: Class Result

[show private | hide private]

Class Result

A 'Result' describes the outcome of a test.

A 'Result' contains two pieces of data: an outcome and a set of annotations. The outcome indicates whether the test passed or failed. More specifically, the outcome may be one of the following constants:

'Result.PASS' -- The test passed.

'Result.FAIL' -- The test failed.

'Result.ERROR' -- Something went wrong in the process of trying to execute the test. For example, if the Python code implementing the 'Run' method in the test class raised an exception, the outcome would be 'Result.ERROR'.

'Result.UNTESTED' -- QMTest did not even try to run the test. For example, if a prerequiste was not satisfied, then this outcome will be used.'

The annotations are a dictionary, mapping strings to strings.

The indices should be of the form 'class.name' where 'class' is the name of the test class that created the annotation. Any annotations created by QMTest, as opposed to the test class, will have indices of the form 'qmtest.name'.

The annotation values are HTML. When displayed in the GUI, the HTML is inserted directly into the result page; when the command-line interface is used the HTML is converted to plain text.

Currently, QMTest recognizes the following built-in annotations:

'Result.CAUSE' -- For results whose outcome is not 'FAIL', this annotation gives a brief description of why the test failed. The preferred form of this message is a phrase like "Incorrect output." or "Exception thrown." The message should begin with a capital letter and end with a period. Most results formatters will display this information prominently.

'Result.EXCEPTION' -- If an exeption was thrown during the test execution, a brief description of the exception.

'Result.TARGET' -- This annotation indicates on which target the test was executed.

'Result.TRACEBACK' -- If an exeption was thrown during the test execution, a representation of the traceback indicating where the exception was thrown.

A 'Result' object has methods that allow it to act as a dictionary from annotation names to annotation values. You can directly add an annotation to a 'Result' by writing code of the form 'result[CAUSE] = "Exception thrown."'.

A 'Result' object is also used to describe the outcome of executing either setup or cleanup phase of a 'Resource'.

Method Summary
	`__init__(self, kind, id, outcome, annotations)` Construct a new 'Result'.
	`__delitem__(self, key)`
	`__getitem__(self, key)`
	`__getstate__(self)` Return a representation of this result for pickling.
	`__setitem__(self, key, value)`
	`__setstate__(self, pickled_state)` Construct a 'Result' from its pickled form.
	`Annotate(self, annotations)` Add 'annotations' to the current set of annotations.
	`CheckExitStatus(self, prefix, desc, status, non_zero_exit_ok)` Check the exit status from a command.
	`Fail(self, cause, annotations)` Mark the test as failing.
	`get(self, key, default)`
	`GetCause(self)` Return the cause of failure, if the test failed.
	`GetId(self)` Return the label for the test or resource.
	`GetKind(self)` Return the kind of result this is.
	`GetOutcome(self)` Return the outcome associated with the test.
	`has_key(self, key)`
	`items(self)`
	`keys(self)`
	`MakeDomNode(self, document)` Generate a DOM element node for this result.
	`NoteException(self, exc_info, cause, outcome)` Note that an exception occurred during execution.
	`Quote(self, string)` Return a version of string suitable for an annotation value.
	`SetCause(self, cause)` Set the cause of failure.
	`SetOutcome(self, outcome, cause, annotations)` Set the outcome associated with the test.

Class Variable Summary
`str`	`CAUSE` = `'qmtest.cause'`
`str`	`END_TIME` = `'qmtest.end_time'`
`str`	`ERROR` = `'ERROR'`
`str`	`EXCEPTION` = `'qmtest.exception'`
`str`	`FAIL` = `'FAIL'`
`list`	`kinds` = `['resource_setup', 'resource_cleanup', 'test']`
`list`	`outcomes` = `['ERROR', 'FAIL', 'UNTESTED', 'PASS']`
`str`	`PASS` = `'PASS'`
`str`	`RESOURCE` = `'qmtest.resource'`
`str`	`RESOURCE_CLEANUP` = `'resource_cleanup'`
`str`	`RESOURCE_SETUP` = `'resource_setup'`
`str`	`START_TIME` = `'qmtest.start_time'`
`str`	`TARGET` = `'qmtest.target'`
`str`	`TEST` = `'test'`
`str`	`TRACEBACK` = `'qmtest.traceback'`
`str`	`UNTESTED` = `'UNTESTED'`

Method Details

init(self, kind, id, outcome='PASS', annotations={})
(Constructor)

Construct a new 'Result'.

'kind' -- The kind of result. The value must be one of the 'Result.kinds'.

'id' -- The label for the test or resource to which this result corresponds.

'outcome' -- The outcome associated with the test. The value must be one of the 'Result.outcomes'.

'annotations' -- The annotations associated with the test.

getstate(self)

Return a representation of this result for pickling.

By using an explicit tuple representation of 'Result's when storing them in a pickle file, we decouple our storage format from internal implementation details (e.g., the names of private variables).

setstate(self, pickled_state)

Construct a 'Result' from its pickled form.

Annotate(self, annotations)

Add 'annotations' to the current set of annotations.

CheckExitStatus(self, prefix, desc, status, non_zero_exit_ok=0)

Check the exit status from a command.

'prefix' -- The prefix that should be used when creating result annotations.

'desc' -- A description of the executing program.

'status' -- The exit status, as returned by 'waitpid'.

'non_zero_exit_ok' -- True if a non-zero exit code is not considered failure.

returns -- False if the test failed, true otherwise.

Fail(self, cause=None, annotations={})

Mark the test as failing.

'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation.

'annotations' -- The annotations are added to the current set of annotations.

GetCause(self)

Return the cause of failure, if the test failed.

returns -- If the test failed, return the cause of the failure, if available.

GetId(self)

Return the label for the test or resource.

returns -- A label indicating indicating to which test or resource this result corresponds.

GetKind(self)

Return the kind of result this is.

returns -- The kind of entity (one of the 'kinds') to which this result corresponds.

GetOutcome(self)

Return the outcome associated with the test.

returns -- The outcome associated with the test. This value will be one of the 'Result.outcomes'.

MakeDomNode(self, document)

Generate a DOM element node for this result.

Note that the context is not represented in the DOM node.

'document' -- The containing DOM document.

returns -- The element created.

NoteException(self, exc_info=None, cause=None, outcome='ERROR')

Note that an exception occurred during execution.

'exc_info' -- A triple, in the same form as that returned from 'sys.exc_info'. If 'None', the value of 'sys.exc_info()' is used instead.

'cause' -- The value of the 'Result.CAUSE' annotation. If 'None', a default message is used.

'outcome' -- The outcome of the test, now that the exception has occurred.

A test class can call this method if an exception occurs while the test is being run.

Quote(self, string)

Return a version of string suitable for an annotation value.

Performs appropriate quoting for a string that should be taken verbatim; this includes HTML entity escaping, and addition of <pre> tags.

'string' -- The verbatim string to be quoted.

returns -- The quoted string.

SetCause(self, cause)

Set the cause of failure.

'cause' -- A string indicating the cause of failure. Like all annotations, 'cause' will be interested as HTML.

SetOutcome(self, outcome, cause=None, annotations={})

Set the outcome associated with the test.

'outcome' -- One of the 'Result.outcomes'.

'cause' -- If not 'None', this value becomes the value of the 'Result.CAUSE' annotation.

'annotations' -- The annotations are added to the current set of annotations.

Class Variable Details

CAUSE

Type:: str
Value:: 'qmtest.cause'

END_TIME

Type:: str
Value:: 'qmtest.end_time'

ERROR

Type:: str
Value:: 'ERROR'

EXCEPTION

Type:: str
Value:: 'qmtest.exception'

FAIL

Type:: str
Value:: 'FAIL'

kinds

Type:: list
Value:: ['resource_setup', 'resource_cleanup', 'test']

outcomes

Type:: list
Value:: ['ERROR', 'FAIL', 'UNTESTED', 'PASS']

PASS

Type:: str
Value:: 'PASS'

RESOURCE

Type:: str
Value:: 'qmtest.resource'

RESOURCE_CLEANUP

Type:: str
Value:: 'resource_cleanup'

RESOURCE_SETUP

Type:: str
Value:: 'resource_setup'

START_TIME

Type:: str
Value:: 'qmtest.start_time'

TARGET

Type:: str
Value:: 'qmtest.target'

TEST

Type:: str
Value:: 'test'

TRACEBACK

Type:: str
Value:: 'qmtest.traceback'

UNTESTED

Type:: str
Value:: 'UNTESTED'

Generated by Epydoc 2.1 on Thu Sep 27 15:49:04 2007

http://epydoc.sf.net