Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7707

[Processor/GCP - PubSub] Avro event re-format when pushed to PubSub

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.11.4
    • Fix Version/s: 1.12.0
    • Component/s: Extensions
    • Labels:

      Description

      Hi, 

      as discussed on Slack : https://apachenifi.slack.com/archives/C0L9VCD47/p1596463216211700

      When we generate an avro event from record and we publish it using PublishPubSub processor the content-type of the file change from avro/binary to UTF-8 due to the following code part : https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-gcp-bundle/nifi-gcp-processors/src/main/java/org/apache/nifi/processors/gcp/pubsub/PublishGCPubSub.java#L157

      This make the event unreadable once Consumed :/ 

      The workaround we found is base64 encoding the avro message, publishing it to PubSub, consuming it, and finally decoding it. 

      We expect this part of encoding would be handled by the processors.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pvillard Pierre Villard
                Reporter:
                aguitton Guitton Alexandre
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h