Parse a list of expressions using pyparsing

喜夏-厌秋 提交于 2021-01-27 21:18:00

问题


I'm trying to use pyparsing to parse simple basic program:

import pyparsing as pp

pp.ParserElement.setDefaultWhitespaceChars(" \t")

EOL = pp.LineEnd().suppress()

# Identifiers is a string + optional $
identifier = pp.Combine(pp.Word(pp.alphas) + pp.Optional("$"))

# Literals (number or double quoted string)
literal = pp.pyparsing_common.number | pp.dblQuotedString

line_number = pp.pyparsing_common.integer

function = pp.Forward()


operand = function | identifier | literal
expression = pp.infixNotation(operand, [
    (pp.oneOf("* / %"), 2, pp.opAssoc.LEFT),
    (pp.oneOf("+ -"), 2, pp.opAssoc.LEFT),
])

assignment = identifier + "=" + expression

# Keywords
PRINT = pp.CaselessKeyword("print")
FOR = pp.CaselessKeyword("for")
TO = pp.CaselessKeyword("to")
STEP = pp.CaselessKeyword("step")
NEXT = pp.CaselessKeyword("next")
CHRS = pp.CaselessKeyword("chr$")

statement = pp.Forward()

print_stmt = PRINT + pp.ZeroOrMore(expression | ";")

for_stmt = FOR + assignment + TO + expression + pp.Optional(STEP + expression)
next_stmt = NEXT

chrs_fn = CHRS + "(" + expression + ")"

function <<= chrs_fn

statement <<= print_stmt | for_stmt | next_stmt | assignment

code_line = pp.Group(line_number + statement + EOL)

program = pp.ZeroOrMore(code_line)

test = """\
10 print 123;
20 print 234; 567;
25 next
30 print 890
"""

print(program.parseString(test).dump())

I do have everything else working except the print clause.

This is the output:

[[10, 'print', 123, ';', 20, 'print', 234, ';', 567, ';', 25, 'next', 30, 'print', 890]]
[0]:
  [10, 'print', 123, ';', 20, 'print', 234, ';', 567, ';', 25, 'next', 30, 'print', 890]

Based on some suggetion I modified my parser but sitll for some reason parser leaks to next row.

How to define list of items for printing properly?


回答1:


There is a subtle thing going on here, and it bears better documenting when using ParserElement.setDefaultWhitespaceChars. This only updates the whitespace characters for pyparsing expressions created after calling setDefaultWhitespaceChars. The built-in expressions like dblQuotedString and the expressions in pyparsing_common are all defined at import time, and so get the standard set of whitespace characters to skip, which includes '\n'. If you create new copies of them using expr.copy() or just plain expr(), you will get new expressions, which use the updated whitespace characters.

Change:

literal = pp.pyparsing_common.number | pp.dblQuotedString
line_number = pp.pyparsing_common.integer

to:

literal = pp.pyparsing_common.number() | pp.dblQuotedString()
line_number = pp.pyparsing_common.integer()

And I think your leakage issues will be resolved.



来源:https://stackoverflow.com/questions/57744290/parse-a-list-of-expressions-using-pyparsing

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!