Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
1.5.0
-
None
Description
PutHiveQL can accept multiple queries separated by a specified delimiter string, ';' by default. It supports users to specify Hive parameters by 'SET' statement. E.g. set 'hive.exec.dynamic.partition.mode'=nonstrict
PutHiveQL also parses each query string with Hive ParseDriver, in order to find input/output table names within queries. However, the aforementioned 'SET' command is not a valid Hive query. The only query can start with 'SET' is 'SET ROLE'. https://raw.githubusercontent.com/apache/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
When set property statement is parsed, following warning message is logged and shown in the NiFi UI:
2018-04-19 05:34:05,616 WARN [Timer-Driven Process Thread-8] o.apache.nifi.processors.hive.PutHiveQL PutHiveQL[id=db408703-0162-1000-0000-000073ad3455] Failed to parse hiveQL: set hive.exec.dynamic.partition.mode=nonstrict due to org.apache.hadoop.hive.ql.parse.ParseException: line 1:4 missing KW_ROLE at 'hive' near 'hive' line 1:8 missing EOF at '.' near 'hive':
In case there are other DML statements such as 'INSERT ...' in the same FlowFile content, those queries are performed successfully regardless of having above parse failure. However, the warning message is mis-leading, it looks as if queries have failed. We should not show such warning message for set property commands.
We can short-circuit query parse logic if statement starts with 'set', since 'set role' does not have any target table. As a reference Hive HCatCli.java has the similar filtering logic.
https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/HCatCli.java#L283
Attachments
Issue Links
- links to
