Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6993

FetchS3 does not prepend user defined S3 metadata keys with s3.<s3 metadtata>

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Component/s: Extensions
    • Labels:

      Description

      S3 allows the assignment of user defined metadata.

      User defined metadata resides in its own "metadata" array and is called by the "getUserMetadata" function of the S3 java SDK [1]

      When NIFI pulls the object and if there is user defined metadata it adds the metadata key / value as its literal key name.

      This could lead to unintended NiFi attributes being replace like "filename"

      A better approach is to prepend the S3 pulled metadata with s3 similar to how Fetchs3 prepends the other metadata of the object with s3.<metadata key>

      [1] https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ObjectMetadata.html#getUserMetadata--

        Attachments

          Activity

            People

            • Assignee:
              digitalplumber Juan C. Sequeiros
              Reporter:
              digitalplumber Juan C. Sequeiros
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: