Tokenizer Class Reference

#include <tokenizer.h>

Collaboration diagram for Tokenizer:

Public Member Functions
	Tokenizer (const Location &loc, QByteArray in)
	Tokenizer (const Location &loc, QFile &file)
	~Tokenizer ()
int	getToken ()
void	setParsingFnOrMacro (bool macro)
const Location &	location () const
QString	previousLexeme () const
QString	lexeme () const
QString	version () const
int	parenDepth () const
int	bracketDepth () const

Static Public Member Functions
static void	initialize ()
static void	terminate ()
	The heap allocated variables are freed here.
static bool	isTrue (const QString &condition)

Protected Attributes
QByteArray	m_in {}
int	m_pos {}

Detailed Description

Definition at line 79 of file tokenizer.h.

Constructor & Destructor Documentation

Tokenizer() [1/2]

Tokenizer::Tokenizer	(	const Location &	loc,
		QByteArray	in )

Definition at line 127 of file tokenizer.cpp.

References m_pos.

Tokenizer() [2/2]

Tokenizer::Tokenizer	(	const Location &	loc,
		QFile &	file )

~Tokenizer()

Tokenizer::~Tokenizer

(

)

Definition at line 134 of file tokenizer.cpp.

Member Function Documentation

bracketDepth()

int Tokenizer::bracketDepth ( ) const

inlinenodiscard

Definition at line 95 of file tokenizer.h.

Referenced by QmlSignatureParser::matchTypeAndName().

Here is the caller graph for this function:

getToken()

int Tokenizer::getToken

(

)

Definition at line 140 of file tokenizer.cpp.

References ignoredTokensAndDirectives, kwordHashTable, KwordHashTableSize, Tok_Ampersand, Tok_Aster, Tok_At, Tok_Caret, Tok_Colon, Tok_Comma, Tok_Comment, Tok_Doc, Tok_Ellipsis, Tok_Eoi, Tok_Equal, Tok_FirstKeyword, Tok_Gulbrandsen, Tok_Ident, Tok_LeftAngle, Tok_LeftBrace, Tok_LeftBracket, Tok_LeftParen, Tok_LeftParenAster, Tok_Number, Tok_RightAngle, Tok_RightBrace, Tok_RightBracket, Tok_RightParen, Tok_Semicolon, Tok_SomeOperator, Tok_String, Tok_Tilde, and Tok_typename.

Referenced by QmlSignatureParser::readToken().

Here is the caller graph for this function:

initialize()

void Tokenizer::initialize ( )

static

Definition at line 482 of file tokenizer.cpp.

References Location::Location(), kwordHashTable, sourceDecoder, Tok_FirstKeyword, and Tok_LastKeyword.

Referenced by processQdocconfFile().

Here is the call graph for this function:

Here is the caller graph for this function:

isTrue()

bool Tokenizer::isTrue ( const QString & condition )

static

Definition at line 723 of file tokenizer.cpp.

References definedX, defines, and falsehoods.

lexeme()

QString Tokenizer::lexeme ( ) const

nodiscard

Definition at line 778 of file tokenizer.cpp.

Referenced by QmlSignatureParser::lexeme().

Here is the caller graph for this function:

location()

const Location & Tokenizer::location ( ) const

inlinenodiscard

Definition at line 90 of file tokenizer.h.

parenDepth()

int Tokenizer::parenDepth ( ) const

inlinenodiscard

Definition at line 94 of file tokenizer.h.

Referenced by QmlSignatureParser::matchParameter().

Here is the caller graph for this function:

previousLexeme()

QString Tokenizer::previousLexeme ( ) const

nodiscard

Definition at line 783 of file tokenizer.cpp.

Referenced by QmlSignatureParser::previousLexeme().

Here is the caller graph for this function:

setParsingFnOrMacro()

void Tokenizer::setParsingFnOrMacro ( bool macro )

inline

Definition at line 88 of file tokenizer.h.

Referenced by QmlSignatureParser::QmlSignatureParser().

Here is the caller graph for this function:

terminate()

void Tokenizer::terminate ( )

static

The heap allocated variables are freed here.

The keyword hash table is not cleared here, but it is cleared in the initialize() function, before any keywords are inserted.

Definition at line 543 of file tokenizer.cpp.

References comment, definedX, defines, falsehoods, ignoredTokensAndDirectives, and versionX.

Referenced by processQdocconfFile().

Here is the caller graph for this function:

version()

QString Tokenizer::version ( ) const

inlinenodiscard

Definition at line 93 of file tokenizer.h.

Member Data Documentation

m_in

QByteArray Tokenizer::m_in {}

protected

Definition at line 173 of file tokenizer.h.

m_pos

int Tokenizer::m_pos {}

protected

Definition at line 174 of file tokenizer.h.

Referenced by Tokenizer().

---

The documentation for this class was generated from the following files:

• qttools/src/qdoc/qdoc/src/qdoc/tokenizer.h (5212645b4665917019a96ba873d9e98c1a136b5f)
• qttools/src/qdoc/qdoc/src/qdoc/tokenizer.cpp (5212645b4665917019a96ba873d9e98c1a136b5f)