[NIFI-4545] Improve Hive processors provenance transit URL

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Extensions
• Assignee: Koji Kawamura
• Reporter: Koji Kawamura
• Created: 30/Oct/17 00:37
• Updated: 18/Nov/19 08:11
• Resolved: 09/Nov/17 17:55

Description

Hive-related processors report NiFi provenance events with transit URLs in a format as 'jdbc:hive2://<host1>:<port1>,<host2>:<port2>/dbName'. The URL format can identify a Hive environment, but not descriptive enough to derive actual table names affecting or being affected by the query which generated the provenance event.

Those table information can only be known by parsing query. This JIRA improves following Hive related processors to write additional 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing Hive queries using Hive parser.

Target Processors:

• PutHiveQL
• SelectHiveQL
• PutHiveStreaming: This processor knows a table name without the need of parsing queries.

Issue Links

• is required by: NIFI-3709 Export NiFi flow dataset lineage to Apache Atlas
• links to: GitHub Pull Request #2239