Class CLexerTOML

• Defined in File lexer_toml.h

Nested Relationships

Nested Types

• Struct CLexerTOML::SToken

Class Documentation

class CLexerTOML

Tokenizes the output of a character reader in regard of the TOML format and returns tokens in order on demand.

Public Types

enum class ETokenCategory : uint8_t

Enum for all possible token categories.

Values:

enumerator token_none

Default.

enumerator token_syntax_assignment

‘=’

enumerator token_syntax_array_open

‘[’ after ‘=’

enumerator token_syntax_array_close

‘]’ after an array open

enumerator token_syntax_table_open

‘[’

enumerator token_syntax_table_close

‘]’

enumerator token_syntax_table_array_open

‘[[’

enumerator token_syntax_table_array_close

‘]]’

enumerator token_syntax_inline_table_open

‘{’

enumerator token_syntax_inline_table_close

‘}’

enumerator token_syntax_comma

‘,’

enumerator token_syntax_dot

‘.’

enumerator token_syntax_new_line

Line break.

enumerator token_key

Key of a Key-Value-Pair.

enumerator token_string

A string for a Value of a Key-Value-Pair or Array.

enumerator token_integer

An integer for a Value of a Key-Value-Pair or Array.

enumerator token_float

A floating point number for a Value of a Key-Value-Pair or Array.

enumerator token_boolean

A bool for a Value of a Key-Value-Pair or Array.

enumerator token_time_local

Unused for now.

enumerator token_date_time_offset

Unused for now.

enumerator token_date_time_local

Unused for now.

enumerator token_date_local

Unused for now.

enumerator token_eof

End of File Token; may only be at the end of the token array.

enumerator token_error

Error token containing an error message; further lexing is not affected.

enumerator token_empty

Empty token for trying to read out of bounds.

enumerator token_terminated

Terminated token containing an error message; further lexing is terminated.

Public Functions

CLexerTOML() = default

Default constructor.

CLexerTOML(const std::string &rssString)

Constructs a new LexerTOML object with given input data that will be lexed.

Parameters:

rssString – [in] The UTF-8 encoded content of a TOML source

void Feed(const std::string &rssString)

Feed the lexer with the given string.

Parameters:

rssString – [in] UTF-8 input string.

void Reset()

Reset the lexer content.

SToken Peek() const

Gets the next token after the current cursor position without advancing the cursor.

Returns:

Returns the next token after the current cursor position or a End-of-File-Token if there are no tokens

SToken Consume()

Gets the next token after the current cursor position and advancing the cursor by one.

Returns:

Returns the next token after the current cursor position or a End-of-File-Token if there are no tokens

SToken Peek(int32_t n)

Gets the n-th token after the current cursor without advancing the cursor.

Parameters:

n – [in] Step size

Returns:

Returns the n-th token after the current cursor position or and empty token if n<1 or the end-token if a step of n would read a position after the end-token

SToken Consume(int32_t n)

Gets the n-th token after the current cursor and advancing the cursor by n.

Parameters:

n – [in] Step size

Returns:

Returns the n-th token after the current cursor position or and empty token if n<1 or the end-token if a step of n would read a position after the end-token

bool IsEnd() const

Checks if the end-token was consumed.

Returns:

Returns true if the end-token was consumed by Consume() or Consume(n) or if there are no tokens; false otherwise

struct SToken

Contains lexed information for the parser.

Public Functions

SToken() = default

Default constructor.

inline explicit SToken(ETokenCategory category)

Constructs a new Token object with a given category.

Parameters:

category – [in] The initial token category value for the token to be constructed

Public Members

std::string ssContentString

Token string content.

int64_t iContentInteger = {}

Token integer content.

double dContentFloatingpoint = {}

Token floatingpoint content.

bool bContentBoolean = {}

Token boolean content.

ETokenCategory eCategory = {ETokenCategory::token_none}

Token category.