Empirical
Public Member Functions | Static Public Member Functions | Static Public Attributes | List of all members
emp::Lexer Class Reference

A lexer with a set of token types (and associated regular expressions) More...

#include <Lexer.h>

Public Member Functions

 Lexer ()
 
 ~Lexer ()
 
size_t GetNumTokens () const
 How many types of tokens can be identified in this Lexer? More...
 
size_t AddToken (const std::string &in_name, const std::string &in_regex)
 Add a new token, specified by a name and the regex used to identify it. More...
 
size_t GetTokenID (const std::string &name) const
 Get the ID associated with a token type (you provide the token name) More...
 
std::string GetTokenName (size_t id) const
 Get the name associated with a token type (you provide the ID) More...
 
TokenInfo GetTokenInfo (const std::string &name) const
 Get the full information about a token (you provide the name) More...
 
void Generate () const
 Create the NFA that will identify the current set of tokens in a sequence. More...
 
Token Process (std::istream &is)
 Get the next token found in an input stream. More...
 
Token Process (std::string &in_str)
 Shortcut to process a string rather than a stream. More...
 
const std::string & GetLexeme ()
 Get the lexeme associated with the last token identified. More...
 
void Print (std::ostream &os=std::cout) const
 Print the full information about this lexer (for debugging) More...
 

Static Public Member Functions

static bool TokenOK (size_t id)
 
static constexpr size_t MaxTokenID ()
 How many total token types are allowed in this lexer? More...
 

Static Public Attributes

static const size_t MAX_TOKEN_ID = 256
 
static const size_t ERROR_ID = MAX_TOKEN_ID
 

Detailed Description

A lexer with a set of token types (and associated regular expressions)

Constructor & Destructor Documentation

emp::Lexer::Lexer ( )
inline
emp::Lexer::~Lexer ( )
inline

Member Function Documentation

size_t emp::Lexer::AddToken ( const std::string &  in_name,
const std::string &  in_regex 
)
inline

Add a new token, specified by a name and the regex used to identify it.

void emp::Lexer::Generate ( ) const
inline

Create the NFA that will identify the current set of tokens in a sequence.

const std::string& emp::Lexer::GetLexeme ( )
inline

Get the lexeme associated with the last token identified.

size_t emp::Lexer::GetNumTokens ( ) const
inline

How many types of tokens can be identified in this Lexer?

size_t emp::Lexer::GetTokenID ( const std::string &  name) const
inline

Get the ID associated with a token type (you provide the token name)

TokenInfo emp::Lexer::GetTokenInfo ( const std::string &  name) const
inline

Get the full information about a token (you provide the name)

std::string emp::Lexer::GetTokenName ( size_t  id) const
inline

Get the name associated with a token type (you provide the ID)

static constexpr size_t emp::Lexer::MaxTokenID ( )
inlinestatic

How many total token types are allowed in this lexer?

void emp::Lexer::Print ( std::ostream &  os = std::cout) const
inline

Print the full information about this lexer (for debugging)

Token emp::Lexer::Process ( std::istream &  is)
inline

Get the next token found in an input stream.

Token emp::Lexer::Process ( std::string &  in_str)
inline

Shortcut to process a string rather than a stream.

static bool emp::Lexer::TokenOK ( size_t  id)
inlinestatic

Member Data Documentation

const size_t emp::Lexer::ERROR_ID = MAX_TOKEN_ID
static
const size_t emp::Lexer::MAX_TOKEN_ID = 256
static

The documentation for this class was generated from the following file: