[NIFI-2435] GetHDFSSequenceFile sequence reader throwing away authenticated filesystem and creating its own

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.1
• Fix Version/s: 1.0.0, 1.0.0-Beta
• Component/s: Core Framework
• Labels: hdfs kerberos
• Environment: Kerberos HDFS
• Assignee: Bryan Rosander
• Reporter: Juan Sequeiros
• Created: 29/Jul/16 17:10
• Updated: 04/Aug/16 21:24
• Resolved: 04/Aug/16 18:03

Description

Hadoop Kerberos has a way of accessing things as an authenticated user
UserGroupInformation.doAs
NIFI is doing that to get access to the filesystem in order to basically tell hadoop to use the correct credentials so our processor does that correctly and then passes the results into the sequence reader unfortunately, the sequence reader then throws away our authenticated filesystem and creates its own without wrapping in a UGI.doAs

More context:
The constructor used for org.apache.hadoop.io.SequenceFile.Reader in both KeyValueReader and ValueReader is deprecated and has been turned into a callthrough that only uses the passed in FileSystem to get the qualified path[1]. I've modified those classes to do that logic and then wrapped the callsite in the processor in a UGI.doAs if it is configured to use a UGI

[1]: https://github.com/apache/hadoop/commit/0c462b223f151208ff7bd5148cee0e436c23d795#diff-e70823f17863445da52875999e4b19a2L1592

Issue Links

• is related to: NIFI-1128 GetHDFSSequenceFile fails with some secure hdfs configurations
• links to: GitHub Pull Request #780
• links to: GitHub Pull Request #787