QAbstractTestLogger Class Reference

Base class for test loggers. More...

#include <qabstracttestlogger_p.h>

Inheritance diagram for QAbstractTestLogger:

Collaboration diagram for QAbstractTestLogger:

Public Types
enum	IncidentTypes {   Skip , Pass , XFail , Fail ,   XPass , BlacklistedPass , BlacklistedFail , BlacklistedXPass ,   BlacklistedXFail }
	\value Pass The test ran to completion successfully. More...
enum	MessageTypes {   QDebug , QInfo , QWarning , QCritical ,   QFatal , Info , Warn }
	The members whose names begin with Q describe messages that originate in calls, by the test or code under test, to Qt logging functions (implemented as macros) whose names are similar, with a q in place of the leading Q. More...

Public Member Functions
	QAbstractTestLogger (const char *filename)
	Constructs the base-class parts of the logger.
virtual	~QAbstractTestLogger ()
	Destroys the logger object.
virtual void	startLogging ()
	Called before the start of a test run.
virtual void	stopLogging ()
	Called after the end of a test run.
virtual void	enterTestFunction (const char *function)=0
	This virtual method is called before each test function is invoked.
virtual void	leaveTestFunction ()=0
	This virtual method is called after a test function has completed, to match \l enterTestFunction().
virtual void	enterTestData (QTestData *)
	This virtual method is called before and after each call to a test function.
virtual void	addIncident (IncidentTypes type, const char *description, const char *file=nullptr, int line=0)=0
	This virtual method is called when an event occurs that relates to the resolution of the test.
virtual void	addBenchmarkResult (const QBenchmarkResult &result)=0
	This virtual method is called after a benchmark has been run enough times to produce usable data.
virtual void	addBenchmarkResults (const QList< QBenchmarkResult > &result)
virtual void	addMessage (QtMsgType, const QMessageLogContext &, const QString &)
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.This virtual method is called from the custom message handler QtTest installs in place of Qt's default message handler for the duration of testing, unless QTest::ignoreMessage() was used to ignore it, or too many messages have previously been processed.
virtual void	addMessage (MessageTypes type, const QString &message, const char *file=nullptr, int line=0)=0
	This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
virtual bool	isRepeatSupported () const
	Returns true if the logger supports repeated test runs.
bool	isLoggingToStdout () const
	Returns true if the output stream is standard output.
void	outputString (const char *msg)
	Convenience method to write msg to the output stream.

Protected Member Functions
void	filterUnprintable (char *str) const
	Helper utility to blot out unprintable characters in str.

Protected Attributes
FILE *	stream

Detailed Description

Base class for test loggers.

\inmodule QtTest

Implementations of logging for QtTest should implement all pure virtual methods of this class and may implement the other virtual methods. This class's documentation of each virtual method sets out how those implementations are invoked by the QtTest code and offers guidance on how the logging class should use the data. Actual implementations may have different requirements - such as a file format with a defined schema, or a target audience to serve - that affect how it interprets that guidance.

Definition at line 30 of file qabstracttestlogger_p.h.

Member Enumeration Documentation

IncidentTypes

enum QAbstractTestLogger::IncidentTypes

\value Pass The test ran to completion successfully.

\value XFail The test failed a check that is known to fail; this failure does not prevent successful completion of the test and may be followed by further checks. \value Fail The test fails. \value XPass A check which was expected to fail actually passed. This is counted as a failure, as it means whatever caused the known failure no longer does, so the test needs an update. \value Skip The current test ended prematurely, skipping some checks. \value BlacklistedPass As Pass but the test was blacklisted. \value BlacklistedXFail As XFail but the test was blacklisted. \value BlacklistedFail As Fail but the test was blacklisted. \value BlacklistedXPass As XPass but the test was blacklisted.

A test may also skip (see \l {QAbstractTestLogger::}{MessageTypes}). The first of skip, Fail, XPass or the blacklisted equivalents of the last two to arise is decisive for the outcome of the test: loggers which should only report one outcome should thus record that as the outcome and either ignore later incidents (or skips) in the same run of the test function or map them to some form of message.

Note

tests can be "blacklisted" when they are known to fail unreliably. When testing is used to decide whether a change to the code under test is acceptable, such failures are not automatic grounds for rejecting the change, if the unreliable failure was known before the change. QTest::qExec(), as a result, only returns a failing status code if some non-blacklisted test failed. Logging backends may reasonably report a blacklisted result just as they would report the non-blacklisted equivalent, optionally with some annotation to indicate that the result should not be taken as damning evidence against recent changes to the code under test.

See also

QAbstractTestLogger::addIncident()

Enumerator
Skip
Pass
XFail
Fail
XPass
BlacklistedPass
BlacklistedFail
BlacklistedXPass
BlacklistedXFail

Definition at line 34 of file qabstracttestlogger_p.h.

MessageTypes

enum QAbstractTestLogger::MessageTypes

The members whose names begin with Q describe messages that originate in calls, by the test or code under test, to Qt logging functions (implemented as macros) whose names are similar, with a q in place of the leading Q.

The other members describe messages generated internally by QtTest.

\value QInfo An informational message from qInfo(). \value QWarning A warning from qWarning(). \value QDebug A debug message from qDebug(). \value QCritical A critical error from qCritical(). \value QFatal A fatal error from qFatal(), or an unrecognised message from the Qt logging functions. \value Info Messages QtTest generates as requested by the {-v1} or {-v2} command-line option being specified when running the test. \value Warn A warning generated internally by QtTest

Note

For these purposes, some utilities provided by QtTestlib as helper functions to facilitate testing - such as \l QSignalSpy, \l QTestAccessibility, \l QTest::qExtractTestData(), and the facilities to deliver artificial mouse and keyboard events - are treated as test code, rather than internal to QtTest; they call \l qWarning() and friends rather than using the internal logging infrastructure, so that \l QTest::ignoreMessage() can be used to anticipate the messages.

See also

QAbstractTestLogger::addMessage()

Enumerator
QDebug
QInfo
QWarning
QCritical
QFatal
Info
Warn

Definition at line 46 of file qabstracttestlogger_p.h.

Constructor & Destructor Documentation

QAbstractTestLogger()

QAbstractTestLogger::QAbstractTestLogger

(

const char *

filename

)

Constructs the base-class parts of the logger.

Derived classes should pass this base-constructor the filename of the file to which they shall log test results, or \nullptr to write to standard output. The protected member stream is set to the open file descriptor.

Definition at line 115 of file qabstracttestlogger.cpp.

~QAbstractTestLogger()

QAbstractTestLogger::~QAbstractTestLogger ( )

virtual

Destroys the logger object.

If the protected stream is not standard output, it is closed. In any case it is cleared.

Definition at line 144 of file qabstracttestlogger.cpp.

Member Function Documentation

addBenchmarkResult()

virtual void QAbstractTestLogger::addBenchmarkResult ( const QBenchmarkResult & result )

pure virtual

This virtual method is called after a benchmark has been run enough times to produce usable data.

It is passed the median result from all cycles of the code controlled by the test's QBENCHMARK loop.

Every logging implementation must implement this method.

See also

addIncident(), addMessage()

Implemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

addBenchmarkResults()

void QAbstractTestLogger::addBenchmarkResults ( const QList< QBenchmarkResult > & result )

virtual

Reimplemented in QPlainTestLogger.

Definition at line 241 of file qabstracttestlogger.cpp.

addIncident()

virtual void QAbstractTestLogger::addIncident ( IncidentTypes type, const char * description, const char * file = nullptr, int line = 0 )

pure virtual

This virtual method is called when an event occurs that relates to the resolution of the test.

The type indicates whether this was a pass, a fail or a skip, whether a failure was expected, and whether the test being run is blacklisted. The description may be empty (for a pass) or a message describing the nature of the incident. Where the location in code of the incident is known, it is indicated by file and line; otherwise, these are nullptr and 0, respectively.

Every logging implementation must implement this method. Note that there are circumstances where more than one incident may be reported, in this way, for a single run of a test on a single dataset. It is the implementation's responsibility to recognize such cases and decide what to do about them. For purposes of counting resolutions of tests in the "Totals" report at the end of a test run, QtTest considers the first incident (excluding XFail and its blacklisted variant) decisive.

See also

addMessage(), addBenchmarkResult()

Implemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

addMessage() [1/2]

virtual void QAbstractTestLogger::addMessage ( MessageTypes type, const QString & message, const char * file = nullptr, int line = 0 )

pure virtual

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

This virtual method is called, via its QtMsgType overload, from the custom message handler QtTest installs. It is also used to warn about various situations detected by QtTest itself, such as failure to see a message anticipated by QTest::ignoreMessage() and, particularly when verbosity options have been enabled via the command-line, to log some extra information.

Every logging implementation must implement this method. The type indicates the category of message and the message is the content to be reported. When the message is associated with specific code, the name of the file and line number within it are also supplied; otherwise, these are \nullptr and 0, respectively.

See also

QTest::ignoreMessage(), addIncident()

Implemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

addMessage() [2/2]

void QAbstractTestLogger::addMessage ( QtMsgType type, const QMessageLogContext & context, const QString & message )

virtual

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.This virtual method is called from the custom message handler QtTest installs in place of Qt's default message handler for the duration of testing, unless QTest::ignoreMessage() was used to ignore it, or too many messages have previously been processed.

(The limiting number of messages is controlled by the -maxwarnings option to a test and defaults to 2002.)

Logging implementations should not normally need to override this method. The base implementation converts type to the matching \l MessageType, formats the given message suitably for the specified context, and forwards the converted type and formatted message to the overload that takes MessageType and QString.

See also

QTest::ignoreMessage(), addIncident()

Reimplemented in QPlainTestLogger.

Definition at line 365 of file qabstracttestlogger.cpp.

enterTestData()

virtual void QAbstractTestLogger::enterTestData ( QTestData * )

inlinevirtual

This virtual method is called before and after each call to a test function.

For a data-driven test, the call before is passed the name of the test data row. This may combine a global data row name with a local data row name. For non-data-driven tests and for the call after a test function, \nullptr is passed

A logging implementation might chose to record the data row name for reporting of results from the test for that data row. It should, in such a case, clear its record of the name when called with \nullptr.

See also

enterTestFunction(), leaveTestFunction()

Reimplemented in QJUnitTestLogger, and QTapTestLogger.

Definition at line 66 of file qabstracttestlogger_p.h.

enterTestFunction()

virtual void QAbstractTestLogger::enterTestFunction ( const char * function )

pure virtual

This virtual method is called before each test function is invoked.

It is passed the name of the test function (without its class prefix) as function. It is likewise called for {initTestCase()} at the start of testing, after \l startLogging(), and for {cleanupTestCase()} at the end of testing, in each case passing the name of the function. It is also called with \nullptr as function after the last of these functions, or in the event of an early end to testing, before \l stopLogging().

For data-driven test functions, this is called only once, before the data function is called to set up the table of datasets and the test is run with its first dataset.

Every logging implementation must implement this method. It shall typically need to record the name of the function for later use in log messages.

See also

leaveTestFunction(), enterTestData()

Implemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

filterUnprintable()

void QAbstractTestLogger::filterUnprintable ( char * str ) const

protected

Helper utility to blot out unprintable characters in str.

Takes a {'\0'}-terminated mutable string and changes any characters of it that are not suitable for printing to {'?'} characters.

Definition at line 179 of file qabstracttestlogger.cpp.

isLoggingToStdout()

bool QAbstractTestLogger::isLoggingToStdout

(

)

const

Returns true if the output stream is standard output.

Definition at line 168 of file qabstracttestlogger.cpp.

isRepeatSupported()

bool QAbstractTestLogger::isRepeatSupported ( ) const

virtual

Returns true if the logger supports repeated test runs.

Repetition of test runs is disabled by default, and can be enabled only for test loggers that support it. Even if the logger may create syntactically correct test reports, log-file analyzers may assume that test names are unique within one report file.

Reimplemented in QPlainTestLogger.

Definition at line 160 of file qabstracttestlogger.cpp.

leaveTestFunction()

virtual void QAbstractTestLogger::leaveTestFunction ( )

pure virtual

This virtual method is called after a test function has completed, to match \l enterTestFunction().

For data-driven test functions, this is called only once, after the test is run with its last dataset.

Every logging implementation must implement this method. In some cases it may be called more than once without an intervening call to \l enterTestFunction(). In such cases, the implementation should ignore these later calls, until the next call to enterTestFunction().

See also

enterTestFunction(), enterTestData()

Implemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

outputString()

void QAbstractTestLogger::outputString

(

const char *

msg

)

Convenience method to write msg to the output stream.

The output msg must be a {'\0'}-terminated string (and not \nullptr). A copy of it is passed to \l filterUnprintable() and the result written to the output stream, which is then flushed.

Definition at line 196 of file qabstracttestlogger.cpp.

Referenced by QCsvBenchmarkLogger::addBenchmarkResult().

Here is the caller graph for this function:

startLogging()

void QAbstractTestLogger::startLogging ( )

virtual

Called before the start of a test run.

This virtual method is called before the first tests are run. A logging implementation might open a file, write some preamble, or prepare in other ways, such as setting up initial values of variables. It can use the usual Qt logging infrastucture, since it is also called before QtTest installs its own custom message handler.

See also

stopLogging()

Reimplemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

Definition at line 222 of file qabstracttestlogger.cpp.

stopLogging()

void QAbstractTestLogger::stopLogging ( )

virtual

Called after the end of a test run.

This virtual method is called after all tests have run. A logging implementation might collate information gathered from the run, write a summary, or close a file. It can use the usual Qt logging infrastucture, since it is also called after QtTest has restored the default message handler it replaced with its own custom message handler.

See also

startLogging()

Reimplemented in QCsvBenchmarkLogger, QJUnitTestLogger, QPlainTestLogger, QTapTestLogger, QTeamCityLogger, QXcodeTestLogger, and QXmlTestLogger.

Definition at line 237 of file qabstracttestlogger.cpp.

Member Data Documentation

stream

FILE* QAbstractTestLogger::stream

protected

Definition at line 87 of file qabstracttestlogger_p.h.

---

The documentation for this class was generated from the following files:

• qtbase/src/testlib/qabstracttestlogger_p.h (80a14c86b2739492d7f7fbdb1cbde1da85d1341d)
• qtbase/src/testlib/qabstracttestlogger.cpp (b5115d1c2fc73feb149a8ee97de011b3c75694fb)