[HIVE-20871] There are some syntax definition wrong in hqlsql.g4

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.1.0, 3.1.1
• Target Version/s: 3.1.1
• Assignee: Unassigned
• Reporter: flying
• Created: 06/Nov/18 08:42
• Updated: 08/Nov/18 04:03

Description

hqlsql.g4 in version 3.x:

1. Any comment in the last line of sql text can not be identified.

sql:

// You will get an error if try parsing it.
select a from b; // I am comment

cause:

// line 1917
L_S_COMMENT : ('--' | '//')  .*? '\r'? '\n' -> channel(HIDDEN) ;       // Single line comment

maybe the code can fix to:

L_S_COMMENT : ('--' | '//')  .*? (('\r'? '\n') | EOF) -> channel(HIDDEN) ;       // Single line comment
// The EOF can be recognized as the end of the single line comment

2. "limit" syntax definition may be mistake.

two sqls:

select a from b limit 10; // just limit 10 record. this sql can be parsed normally.

select a from b limit 10, 10; // pagnation, 10 record per page. you will get an error if try parsing it.

cause:

// line 900
select_options_item :
       T_LIMIT expr
     | T_WITH (T_RR | T_RS | T_CS | T_UR) (T_USE T_AND T_KEEP (T_EXCLUSIVE | T_UPDATE | T_SHARE) T_LOCKS)?
     ;

maybe the code can fix to:

// line 900
select_options_item :
       T_LIMIT expr (T_COMMA expr)?
     | T_WITH (T_RR | T_RS | T_CS | T_UR) (T_USE T_AND T_KEEP (T_EXCLUSIVE | T_UPDATE | T_SHARE) T_LOCKS)?
     ;
// add optional '(T_COMMA expr)?' after 'T_LIMIT expr'.

3. The word 'type' can not be recognized as a column name.

sql:

select type from someTable;
// 'type' is a column. you will get an error if try parsing it. but you can execute the sql in hive query engine normally.

Cause the lexer 'T_TYPE' is not added to 'non_reserved_words'.

// Sorry for not good at English. Talk is cheap, just show you code.