Function Tokenize(const char *, const CContextPtr&)

• Defined in File lexer.h

Function Documentation

CTokenList Tokenize(const char *szCode, const CContextPtr &rptrContext)

Tokenize a string into a token list.

Parameters:

• szCode – [in] The code to tokenize.

• rptrContext – [in] Reference to the smart pointer to the source code context.

Returns:

The tokenized string.