我对antlr 4 Lexer.getCharPositionInLine()函数的理解是它应该返回“从零开始计算令牌的第一个字符的行内的字符位置” - The Definitive Antlr 4 Reference
使用以下antlr 4语法,似乎Lexer函数getCharPositionInLine()始终返回0.请注意COMMENT词法分析器规则中的Java代码 . 它包含用于打印从getCharPositionInLine()返回的值的代码 .
grammar Expr;
compilUnit : stat+ EOF ;
stat : assign NEWLINE ;
assign : IDENT ASSIGN INT ;
// Lexer rules
ASSIGN : '=' {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/");} ;
APOS : '\'' {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/");} ;
INT : ('0' | '-'? [1-9][0-9]*) {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/");} ;
IDENT : [a-zA-Z][a-zA-Z0-9]* {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/");} ;
/* For lines that have only a comment preceeded by optional white space,
* skip the entire line including the newline. For lines that have a
* comment preceeded by other code skip the comment and return a
* NEWLINE. */
COMMENT : [ \t]* APOS NEND* END
{
int line = getLine();
int pos = getCharPositionInLine();
System.out.println("COMMENT " + line + ":" + pos + " /" + getText() + "/");
if (pos == 0) {
skip();
}
else {
setType(NEWLINE);
setText("\n");
}
}
;
NEWLINE : END+ {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/");} ;
WS : [ \t]+ {System.out.println(getLine() + ":" + getCharPositionInLine() + " /" + getText() + "/"); skip();} ;
fragment END : '\u000c'? '\r'? '\n' ;
fragment NEND : ~[\u000c\r\n] ;
我从命令行使用这三个命令:
java -jar antlr/antlr-4.1-complete.jar Expr.g4
javac -cp antlr/antlr-4.1-complete.jar Expr*.java
java -cp "antlr/antlr-4.1-complete.jar;." org.antlr.v4.runtime.misc.TestRig Expr compilUnit -tokens progs/hello.laf
并为此输入:
'Yo
x = 3 'Yay
我得到这个输出:
COMMENT 2:0 /'Yo
/
2:1 /x/
2:2 / /
2:3 /=/
2:4 / /
2:5 /3/
COMMENT 3:0 / 'Yay
/
[@0,4:4='x',<4>,2:0]
[@1,6:6='=',<1>,2:2]
[@2,8:8='3',<3>,2:4]
[@3,16:15='<EOF>',<-1>,3:0]
line 3:0 missing NEWLINE at '<EOF>'
似乎因为COMMENT词法分析器规则包括匹配换行符已经将行号加1并将字符位置重置为0的换行符 . 但是,这与“The Definitive Antlr 4 Reference”中的文档不匹配 . 说 . 我究竟做错了什么?或者这是Antlr 4中的错误?
1 回答
您将Token.getCharPositionInLine()与Lexer.getCharPositionInLine()混淆 . 后者返回当前词法分析器位置,对于您的操作,该位置显然始终为0,因为您的操作立即放置在所需的换行符之后 .