Flex: How to define a term to be the first one at the beginning of a line(exclusively)

问题

I need some help regarding a problem I face in my flex code.

My task: To write a flex code which recognizes the declaration part of a programming language, described below.

Let a programming language PL. Its variable definition part is described as follows:

At the beginning we have to start with the keyword "var". After writing this keyword we have to write the variable names(one or more) separated by commas ",". Then a colon ":" is inserted and after that we must write the variable type(say real, boolean, integer or char in my example) followed by a semicolon ";". After doing the previous steps there is the potentiality to declare into a new line new variables(variable names separated by commas "," followed by colon ":" followed by variable type followed by a semicolon ";"), but we must not use the "var" keyword again at the beginning of the new line( the "var" keyword is written once!!!)

E.g.

var number_of_attendants, sum: integer;
ticket_price: real;
symbols: char;

Concretely, I do not know how to make it possible to define that each and every declaration part must start only with the 'var' keyword. Until now, if I would begin a declaration part directly declaring a variable, say x (without having written "var" at the beginning of the line), then no error would occur(unwanted state).

My current flex code below:

%{
#include <stdio.h>
%}
VAR_DEFINER "var"
VAR_NAME [a-zA-Z][a-zA-Z0-9_]*
VAR_TYPE "real"|"boolean"|"integer"|"char"
SUBEXPRESSION [{VAR_NAME}[","{VAR_NAME}]*":"[ \t\n]*{VAR_TYPE}";"]+
EXPRESSION {VAR_DEFINER}{SUBEXPRESSION}
%%
^{EXPRESSION}                 { 
                                  printf("This is not a well-syntaxed expression!\n"); 
                                  return 0;
                            }
{EXPRESSION}                        printf("This is a well-syntaxed expression!\n");
";"[ \t\n]*{VAR_DEFINER}    {
                                  printf("The keyword 'var' is defined once at the beginning of a new line. You can not use it again\n");
                                  return 0;
                            }
{VAR_DEFINER}                  printf("A keyword: %s\n", yytext);
^{VAR_DEFINER}                 printf("Each and every declaration part must start with the 'var' keyword.\n");
{VAR_TYPE}";"                     printf("The variable type is: %s\n", yytext);
{VAR_NAME}                        printf("A variable name: %s\n", yytext);
","/[ \t\n]*{VAR_NAME}            /* eat up commas */
":"/[ \t\n]*{VAR_TYPE}";"         /* eat up single colon */
[ \t\n]+                          /* eat up whitespace */
.                           {
                                  printf("Unrecognized character: %s\n", yytext);
                                  return 0;
                            }
%%
main(argc, argv)
int argc;
char** argv;
{
++argv, --argc;
if (argc > 0) 
yyin = fopen(argv[0],"r");
else 
yyin = stdin;
yylex();
}

I hope to have made it as much as possible clear.

I am looking forward to reading your answers!

回答1:

You seem to be trying to do too much in the scanner. Do you really have to do everything in Flex? In other words, is this an exercise to learn advanced use of Flex, or is it a problem that may be solved using more appropriate tools?

I've read that the first Fortran compiler took 18 staff-years to create, back in the 1950's. Today, "a substantial compiler can be implemented even as a student project in a one-semester compiler design course", as the Dragon Book from 1986 says. One of the main reasons for this increased efficiency is that we have learned how to divide the compiler into modules that can be constructed separately. The two first such parts, or phases, of a typical compiler is the scanner and the parser.

The scanner, or lexical analyzer, can be generated by Flex from a specification file, or constructed otherwise. Its job is to read the input, which consists of a sequence of characters, and split it into a sequence of tokens. A token is the smallest meaningful part of the input language, such as a semicolon, the keyword var, the identifier number_of_attendants, or the operator <=. You should not use the scanner to do more than that.

Here is how I woould write a simplified Flex specification for your tokens:

[ \t\n] { /* Ignore all whitespace */ }
var { return VAR; }
real { return REAL; }
boolean { return BOOLEAN; }
integer { return INTEGER; }
char { return CHAR; }
[a-zA-Z][a-zA-Z0-9_]* { return VAR_NAME; }
. { return yytext[0]; }

The sequence of tokens is then passed on to the parser, or syntactical analyzer. The parser compares the token sequence with the grammar for the language. For example, the input var number_of_attendants, sum : integer; consists of the keyword var, a comma-separated list of variables, a colon, a data type keyword, and a semicolon. If I understand what your input is supposed to look like, perhaps this grammar would be correct:

program : VAR typedecls ;
typedecls : typedecl | typedecls typedecl ;
typedecl : varlist ':' var_type ';' ;
varlist : VAR_NAME | varlist ',' VAR_NAME ;
var_type : REAL | BOOLEAN | INTEGER | CHAR ;

This grammar happens to be written in a format that Bison, a parser-generator that often is used together with Flex, can understand.

If you separate your solution into a lexical part, using Flex, and a grammar part, using Bison, your life is likely to be much simpler and happier.

来源：https://stackoverflow.com/questions/34375410/flex-how-to-define-a-term-to-be-the-first-one-at-the-beginning-of-a-lineexclus

标签

flex-lexer

lex