lexer

package

v1.0.0 Latest Latest Go to latest Published: Jan 16, 2025 License: MIT Imports: 2 Imported by: 0

Details

Package lexer is converting a sequence of characters (stdin or input file) into a sequence of lexical tokens, which are later interpreted by parser.

This section is empty.

This section is empty.

This section is empty.

type Lexer struct {
	// contains filtered or unexported fields
}

Lexer - Represent the text input (commands) provided by client before tokenization

func RunLexer(input string) *Lexer

RunLexer - Start lexer by moving to the first position

func (lexer *Lexer) NextToken() token.Token

NextToken - Return the next token structure which is appearing after current position.