Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5876

PutHDFS executes with SUCESS, although exception in logs and no file saved.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Not A Bug
    • Affects Version/s: 1.7.1
    • Fix Version/s: 1.8.0
    • Component/s: None
    • Labels:
    • Environment:
      NiFi on Windows 10

      Description

      Issue:
      PutHDFS executes with SUCESS, although exception in logs and no file saved.

      Solution:
      Propagate exception to UI (with FAILURE status).

      Steps to reproduce

      1. This error applies to an installation of Nifi on a MS Windows system
      2. Make your own application properties file (e.g. myapp-nifi.properties) and link it in the main nifi.properties file (with property “nifi.variable.registry.properties=<mypath>/myapp-nifi.properties”)
      3. Enter a new property “hadoop_conf=C:\bin\hadoop-2.7.1\etc\hadoop\core-site.xml,C:\bin\hadoop-2.7.1\etc\hadoop\hdfs-site.xml” in myapp-nifi.properties (Hadoop is installed in the corresponding directory)
      4. The backslashes of the directory path of the configuration are the important thing to look at
      5. Restart Nifi to take the configuration change into account
      6. Open the properties configuration of the PutHDFS processor
      7. Set the " Hadoop Configuration Resources" to the variable key of the external property: ${hadoop_conf}.
      8. Execute the flow/processor with some path and filename to put into HDFS.
      9. Processor executes with SUCCESS
      10. Log file shows "java.lang.reflect.InvocationTargetException: null"
      11. No file is written into HDFS

      Example external properties file:
      hadoop_conf=C:\bin\hadoop-2.7.1\etc\hadoop\core-site.xml,C:\bin\hadoop-2.7.1\etc\hadoop\hdfs-site.xml

      Setting the path value directly in the PutHDFS processor with the "\" path can be used and executes correct (no error, file is stored in HDFS).

      Workaround:
      Replace windows backslah "\" with "/" in the external property file (e.g. hadoop_conf=C:/bin/hadoop-2.7.1/etc/hadoop/core-site.xml,C:/bin/hadoop-2.7.1/etc/hadoop/hdfs-site.xml).

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              preja Jan-Henrik Preuss
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: