Writing re-entrant lexer with Flex

為{幸葍}努か 提交于 2019-12-01 05:11:46

For a reentrant lexer, all communication must include the state, which is contained within the scanner.

Anywhere in your program (e.g. inside main) you can access the state variables via special functions to which you will pass your scanner. E.g., in your original reentrant.l, you can do this:

yyscan_t scanner;
yylex_init(&scanner);
yyset_in(fopen(argv[1], "rb"), scanner);
yylex(scanner);
yylex_destroy(scanner);

I have renamed scanner to avoid confusion with yyscanner in the actions. In contrast with general C code, all your actions occur within a giant function called yylex, which is passed your scanner by the name yyscanner. Thus, yyscanner is available to all your actions. In addition, yylex has a local variable called yyg that holds the entire state, and most macros conveniently refer to yyg.

While it is true that you can use the yyin macro inside main by defining yyg as you did in your own Answer, that is not recommended. For a reentrant lexer, the macros are meant for actions only.

To see how this is implemented, you can always view the generated code:


/* For convenience, these vars
   are macros in the reentrant scanner. */
#define yyin yyg->yyin_r
...

/* Holds the entire state of the reentrant scanner. */
struct yyguts_t
...

#define YY_DECL int yylex (yyscan_t yyscanner)

/** The main scanner function which does all the work.
 */
YY_DECL
{
    struct yyguts_t * yyg = (struct yyguts_t*)yyscanner;
...
}

There is lots more on the reentrant option in the flex docs, which include a cleanly compiling example. (Google "flex reentrant", and look for the flex.sourceforge link.) Unlike bison, flex has a fairly straight-forward model for reentrancy. I strongly suggest using reentrant flex with Lemon Parser, rather than with yacc/bison.

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!