Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-710

Enumerate available Hadoop library versions for GetHDFS and PutHDFS

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Extensions
    • Labels:
    • Environment:
      Unix, Hadoop

      Description

      As far as I'm aware, it is only possible to use a single Hadoop library version per NiFi instance/cluster.

      Would it be possible in some way to enumerate the available Hadoop library versions in the /lib directory and give the user the option to select which version of Hadoop cluster they are writing to or reading from per HDFS processor? The intent would be to allow a single NiFi instance/cluster to write to multiple HDFS clusters which may be running on different versions eg. QA and Prod. This eliminates the need for having multiple HDFS feeder NiFis only because the Hadoop versions are different.

      I would try and figure this out my self however I don't know a whole lot about class loading in NiFi or generally how I would start this.

        Attachments

          Activity

            People

            • Assignee:
              ozhurakousky Oleg Zhurakousky
              Reporter:
              NathanG Nathan Gough
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: