Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3068

NiFi can not reliably support multiple HDFS clusters in the same flow

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: Core Framework
    • Labels:

      Description

      The HDFS configurations in PutHDFS are not respected when two (or more) PutHDFS processors exist with different configurations. The second processor to run will use the configurations from the first processor. This can cause data to be written to the wrong cluster.

      This appears to be caused by configuration caching in AbstractHadoopProcessor, which would affect all HDFS processors.
      https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/AbstractHadoopProcessor.java#L144

        Attachments

        1. NIFI-3068.patch
          2 kB
          Bryan Bende

          Issue Links

            Activity

              People

              • Assignee:
                bbende Bryan Bende
                Reporter:
                samhjelmfelt Sam Hjelmfelt
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: