Function Tokenize(const std::string&, const CContextPtr&)

• Defined in File lexer.h

Function Documentation

CTokenList Tokenize(const std::string &rssCode, const CContextPtr &rptrContext)

Tokenize a string into a token list.

Parameters:

• rssCode – [in] The code to tokenize.

• rptrContext – [in] Reference to the smart pointer to the source code context.

Returns:

The tokenized string.