Skip to content

lookahead without modifying the lexer state? #7

@NicholasBHubbard

Description

@NicholasBHubbard

I am using lexer along with your other project parse.

I am having trouble figuring out how I can lookahead on my input without modifying the lexer state.

In this example I wish for check-bar to be able to lookahead to see if the next token is "bar" and if it is to just ignore it without consuming the token and fail otherwise.

(define-lexer my-lexer (st)
  ("%s+" (values :next-token $$))
  ("foo" (values :foo $$))
  ("bar" (values :bar $$)))

(define-parser check-bar
  "This parser will fail unless the next token is bar but will not consume the token."
  (.let (state0 (.get))
    (.do (.is :bar)
         (.put state0))))

(define-parser foo-then-bar
  (.let* ((foo (.is :foo))
          (_ 'check-bar)
          (bar (.is :bar)))
    (.ret (list foo bar))))

(defun run-parser (string)
  (with-lexer (lexer 'my-lexer string)
    (with-token-reader (next-token lexer)
      (parse 'foo-then-bar next-token))))

However the following repl output shows that check-bar consumes token from the lexer state:

PARSER> (run-parser "foo bar")
; Debugger entered on #<LEXER::LEX-ERROR {1004E95393}>
[1] PARSER> 
; Evaluation aborted on #<LEXER::LEX-ERROR {1004E95393}>
PARSER> (run-parser "foo bar bar")
("foo" "bar")
T

How can I lookahead at my input without consuming any tokens?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions