[NIFI-4544] Improve HDFS processors provenance transit URL

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Extensions
• Assignee: Koji Kawamura
• Reporter: Koji Kawamura
• Created: 30/Oct/17 00:29
• Updated: 02/Nov/17 09:11
• Resolved: 02/Nov/17 09:11

Description

HDFS related processors report NiFi provenance events with transit URLs in a format as a Path in HDFS, e.g. '/user/nifi/21495865474943'. However, the URL is not descriptive enough if a NiFi interacts with multiple HDFS environments having the same paths.

HDFS processors transit URL should include host information it operates with, so that an URL can identify a HDFS environment. Also, those URLs should have 'hdfs://' protocol so that external programs can understand a provenance event is performed against HDFS.

Target Processors:

• FetchHDFS
• FetchParquet
• GetHDFS
• GetHDFSSequenceFIle
• PutHDFS
• PutParquet

Issue Links

• is required by: NIFI-3709 Export NiFi flow dataset lineage to Apache Atlas
• links to: GitHub Pull Request #2238