Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
The following is a list of improvements/fixes to attribute handling in PutHDFS for consideration:
1) The documentation says that PutHDFS writes the "filename" attribute with the description: "The name of the file written to HDFS comes from the value of this attribute." PutHDFS should add a @ReadsAttribute annotation with the above description.
2) The @WritesAttribute description above should read something like: The name of the file written to HDFS is stored in this attribute." Also it doesn't appear by the code that this is accurate; the filename could've changed depending on the compression option, but it appears the original filename attribute value is retained
3) PutHDFS should write an attribute (such as absolute.path like GetFile) that contains the value of the Directory property after any Expression Language has been evaluated. This would be documented via @WritesAttribute.
#2 and #3 are being done for the Transit URI in the reported provenance event(s), this should be extended to the flow file attributes as well.
Attachments
Issue Links
- links to
