更新
所以问题似乎与生成器有关，而不一定与next-token和lookahead函数有关。我在set！s发生的地方添加了一些显示调用，并发现问题是第二次调用（generate-token）之后，它会从第一次调用它的地方恢复执行。
以下是该计划的完整代码（我已将以下原始帖子留待参考）：
(define char-alphanumeric? (lambda (char) (or (char-alphabetic? char) (char-numeric? char))))
(define generate-token #f)
(define filename "input.txt")
(define next-token #f)
(define lookahead #f)
(define status #f)
(let ((f (open-input-file filename)) (yield #f) (token "") (lookahead-token #f) (current-token #f))
(set! generate-token (lambda ()
(letrec ((next-char (lambda (c)
(let ((separators (list #\; #\,)))
(cond ((eof-object? c) (display "last token before eof: ") (display token) (newline) (yield c))
((member c separators)
(begin
(display "token before sep: ") (display token) (newline)
(call-with-current-continuation (lambda (resume)
(set! generate-token (lambda () (resume)))
(yield token)))
(display "back from call") (set! token "")
(call-with-current-continuation (lambda (resume)
(set! generate-token (lambda () (resume)))
(yield (make-string 1 c))))
))
((or (char-alphanumeric? c) (equal? c #\_)) ; c is part of a string token
(begin (display "found char: ") (display c) (display "; added to string: ")
(set! token (string-append token (make-string 1 c)))
(display token) (newline)
(next-char (read-char f))))
((char-whitespace? c)
(begin
(display "token before ws: ") (display token) (newline)
(if (> (string-length token) 0)
(begin (call-with-current-continuation (lambda (resume)
(display "setting generate-token to resume") (newline)
(set! generate-token (lambda ()
((display "calling resume") (newline)
(resume))))
(display "yielding token from cc") (newline)
(yield token)))
(display "continuing...") (newline)
(set! token ""))
;(set! token "")
))))
(next-char (read-char f))
))))
(call-with-current-continuation (lambda (k) ((set! yield k) (k (next-char (read-char f))))))
)))
(set! lookahead (lambda () (begin
(if (not lookahead-token)
(begin (display "no lookahead") (newline)
(display "setting lookahead-token") (newline)
(set! lookahead-token (string-copy (generate-token)))
(display "lookahead set to ") (display lookahead-token) (newline)
))
lookahead-token)))
(set! next-token (lambda () (begin
(if lookahead-token
(begin (display "affirmative") (newline)
(set! current-token (string-copy lookahead-token))
(set! lookahead-token #f))
(begin (display "negative") (newline)
(display "setting current token to next-token") (newline)
(set! current-token (string-copy (generate-token)))
(display "current token = ") (display current-token) (newline)
(set! lookahead-token #f)))
current-token)))
(set! status (lambda () (begin (display current-token) (display " -> ") (display lookahead-token) (newline))))
)
根据以下原始帖子中的第一个示例执行next-token和lookahead调用会产生：
> (next-token)
negative
setting current token to next-token
found char: t; added to string: t
found char: h; added to string: th
found char: e; added to string: the
found char: s; added to string: thes
found char: e; added to string: these
token before ws: these
setting generate-token to resume
yielding token from cc
current token = these
"these"
> (status)
these -> #f
> (lookahead)
no lookahead
setting lookahead-token
calling resume
continuing...
found char: a; added to string: a
found char: r; added to string: ar
found char: e; added to string: are
token before ws: are
setting generate-token to resume
yielding token from cc ; the problem is right here: the generate token call is
current token = are ; sending control back to next-token instead of lookahead.
"are"
> (status)
are -> #f
我不知道为什么它会以这种方式行事，但我会承认我是新手，并且可能不完全了解其后果。任何帮助，一如既往，将不胜感激。
感谢。
原帖如下：
我创建了一个生成器，它解析一个文本文件并一次返回一个令牌作为字符串。所以，如果我有一个包含
的文件these are my file contents
对（生成 - 令牌）的连续调用分别返回“这些”“是”“我的”。这似乎是有效的，但我把它作为更大任务的解析器的一部分写了。生成器似乎工作顺利，但是当我构建一个LR（1）解析器来解析令牌流时，我需要能够执行前瞻。为此，我创建了以下程序：
(define generate-token #f)
(define next-token #f)
(define lookahead #f)
(define status #f)
(let ((lookahead-token #f) (current-token #f))
(set! generate-token (lambda () ... ) ; the generator function
(set! lookahead (lambda () (begin
(if (not lookahead-token)
(begin (display "no lookahead") (newline)
(set! lookahead-token (string-copy (generate-token)))))
lookahead-token)))
(set! next-token (lambda () (begin
(if lookahead-token
(begin (display "affirmative") (newline)
(set! current-token (string-copy lookahead-token))
(set! lookahead-token #f))
(begin (display "negative") (newline)
(set! current-token (string-copy (generate-token)))
(set! lookahead-token #f)))
current-token)))
(set! status (lambda () (begin (display current-token) (display " -> ") (display lookahead-token) (newline))))
)
但是，这些不能按预期工作。我的印象是scheme（这是用drRacket编写的，但使用#lang r5rs）按值传递对象，所以（字符串复制调用假设不需要，但这仍然不能按预期工作。它的工作方式如此：
> (status)
#f -> #f
> (next-token)
"these"
> (status) ; next-token properly sets current-token
"these" -> #f
> (lookahead) ; generator returns "are" as expected
"are"
> (status) ; notice that the current-token has been replaced instead of the lookahead-token
"are" -> #f
在不同的流程中，如果首先调用（lookahead），它可以正常工作。
> (status)
#f -> #f
> (lookahead)
"these"
> (status)
#f -> "these"
> (lookahead)
"these"
> (status)
#f -> "these"
> (next-token)
"these"
> (status)
"these" -> #f
> (lookahead)
"are"
> (status)
"these" -> "are"
如果有人知道发生了什么，我们将非常感谢任何见解。披露：这是学校工作，但我不是要求你为我做这些＆gt;。＆gt;。
答案 0 :(得分：0)
问题是您yield的实施。以下是您的可读性实现的简化版本：
(define yield #f)
(define my-generator
(lambda ()
(let forever ((num 0))
(let/cc resume
(set! my-generator resume)
(yield num))
(forever (add1 num)))))
(define zero (let/cc my-yield
(set! yield my-yield)
(my-generator)))
(define one (my-generator))
zero
;; => 1
延续总是跳回到其返回值最初的位置。因此，如果您使用yield或call/cc抓取let/cc，那么每次yield，您都会跳回到捕获它的位置。
在您的代码中，您花了一些时间在每个yield之前更新简历续订，但您只在一个您关注的地方更新yield
致电next-token。您每次打电话yield时都需要更新generate-token。
对于那些没有在校学习的读者，只需使用racket/generator，其中所有的工作都是为您完成的。