flex-lexer | 易学教程

Flex: How to define a term to be the first one at the beginning of a line(exclusively)

阅读更多关于 Flex: How to define a term to be the first one at the beginning of a line(exclusively)

问题 I need some help regarding a problem I face in my flex code. My task: To write a flex code which recognizes the declaration part of a programming language, described below. Let a programming language PL. Its variable definition part is described as follows: At the beginning we have to start with the keyword "var". After writing this keyword we have to write the variable names(one or more) separated by commas ",". Then a colon ":" is inserted and after that we must write the variable type(say

Bison does not appear to recognize C string literals appropriately

阅读更多关于 Bison does not appear to recognize C string literals appropriately

My problem is that I am trying to run a problem that I coded using a flex-bison scanner-parser. What my program is supposed to do is take user input (in my case, queries for a database system I'm designing), lex and parse, and then execute the corresponding actions. What actually happens is that my parser code is not correctly interpreting the string literals that I feed it. Here's my code: 130 insertexpr : "INSERT" expr '(' expr ')' 131 132 { 133 $$ = new QLInsert( $2, $4 ); 134 } 135 ; And my input, following the "Query: " prompt: Query: INSERT abc(5); input:1.0-5: syntax error, unexpected

Resetting the state of flex and/or bison

阅读更多关于 Resetting the state of flex and/or bison

问题 As part of a toy project I've been trying to make a small modification of someone else's parser based on flex/bison. I'm really not experienced with either. You can find the original parser here. I've been trying to put together a simple function that accepts a string and returns a parse tree, so I can expose this via FFI for use in another programming language. What I have is mostly based on the main() function in the original program, my butchered version is below: TreeNode* parse_string

Why does Flex say this is an “unrecognized rule”?

阅读更多关于 Why does Flex say this is an “unrecognized rule”?

问题 In the following: space ([ \t\f\r])+ opt_space ([ \t\f\r])* cpp ^{opt_space}#{opt_space} word [A-Za-z_][A-Za-z_0-9]* arg_macro {cpp}define{space}{word} /*arg_macro ^{opt_space}#{opt_space}define{space}{word}*/ %% {arg_macro} ; %% I get an error message test.l:9: unrecognized rule If I uncomment the second version of arg_macro and comment the first one, the error message goes away. Any ideas why? 回答1: If you remove the ^ from the cpp definition, and place it in the arg_macro definition, then

Integrating Flex/Bison with external program

阅读更多关于 Integrating Flex/Bison with external program

问题 I'm working on an intelligent agent model that requires, as input, a list of events. The events come from the output of another model and are in a (large) text file. The text file is a list of all events (including unnecessary events that I don't care about), so I've written a scanner using flex that can find the useful bits. The framework for the intelligent agent model is already written in C++. Each event is timestamped and contains a large amount of information about the event. The format

Why is yylval null?

阅读更多关于 Why is yylval null?

问题 I'm trying to write my first parser with Flex & Bison. When parsing numbers, I'm trying to save their values into the yylval structure. The problem is, yylval is null when the lexer reaches a number, which causes a segmentation fault. (Related point of confusion: why is it that in most Flex examples (e.g. here), yylval is a structure, rather than a pointer to a structure? I couldn't get yylval to be recognized in test.l without %option bison-bridge , and that option made yylval a pointer.

What is the meaning of yytext[0]?

阅读更多关于 What is the meaning of yytext[0]?

问题 What is the meaning of yytext[0]? And why should we use in the lex and yacc program? I'm learner so don't mind if it is a silly question. 回答1: yytext holds the text matched by the current token. So yytext[0] holds the first character of the text matched by the current token. Sometimes you have a rule which can match different texts so you need to get the real text matched like for variable names or you have a rule to match all arithmetic operations. A good source is the Flex manual. For

Flex rule with a period “.” is not compiling

阅读更多关于 Flex rule with a period “.” is not compiling

问题 I am facing a problem compiling this regular expression with flex "on"[ \t\r]*[.\n]{0,300}"."[ \t\r]*[.\n]{0,300}"from" {counter++;} I had 100 hundred rules in rules section of flex specification file. I tried to compile it flex -Ce -Ca rule.flex I waited for 10 hours still it didn't complete so I killed it. I started to find the issue and narrowed down the problem to this rule. If I remove this rule from 100 rules, it takes 21 seconds to compile it to C code. If I replace the period with

Difficulties during the compilation (g++, bison, flex) with yyparse();

阅读更多关于 Difficulties during the compilation (g++, bison, flex) with yyparse();

问题 I have a problem with compilation of my code: Flex: %{ #include "lista4.tab.hpp" #include <stdlib.h> extern int yylex(); %} %% "=" {return EQ;} "!=" {return NE;} "<" {return LT;} ">" {return GT;} ":=" {return ASSIGN;} ";" {return SEMICOLON;} "IF" {return IF;} "THEN"{return THEN;} "END" {return END;} [_a-z]+ {yylval.text = strdup(yytext); return IDENTIFIER;} [ \t]+ [0-9]+ { yylval.var = atoi (yytext); return NUMBER; } [-+/^*'%'()] { return *yytext; } \n return RESULT; %% Bison: %{ extern "C" {

How is maximal-munch implemented?

阅读更多关于 How is maximal-munch implemented?

问题 I am studying compilers and am learning about lexical analysis. I understand that one specifies each lexeme as a regular expression, and using flex , a lexer can be automatically generated. I am further learning about how the regular expression is converted to an NFA which is then converted to a DFA, where it can be quickly evaluated. However, my question is, how is the maximal-munch rule implemented? Internally, how does the lexer know to "keep going" to find the longest possible lexeme?