Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5010

putparquet processor in Nifi 1.5 not working

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: Configuration
    • Labels:
      None
    • Environment:
      x86_64 GNU/Linux

      Description

      putpaquet was working in 1.3 version and is not working in 1.5 version after the upgrade.

      I do know that the below mentioned class JniBasedUnixGroupsMapping is available in libarary hadoop-common which i can see under nifi-1.5.0/work/nar/extensions/nifi-hadoop-libraries-nar-1.5.0.nar-unpacked/META-INF/bundled-dependencies/hadoop-common-2.7.3.jar. 

      I tried to put this folder in the classpath as well but still no luck

      simple json saving to hdfs using putparquet is failing with below error

      Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.JniBasedUnixGroupsMapping
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:348)
      at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor$ExtendedConfiguration.getClassByNameOrNull(AbstractHadoopProcessor.java:490)
      at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2099)
      at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
      at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2219)
      at org.apache.hadoop.security.Groups.<init>(Groups.java:80)
      at org.apache.hadoop.security.Groups.<init>(Groups.java:74)
      at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:303)
      at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:284)
      at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:312)
      at org.apache.nifi.hadoop.SecurityUtil.loginKerberos(SecurityUtil.java:67)
      at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:275)
      at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:205)
      at org.apache.nifi.processors.hadoop.AbstractPutHDFSRecord.onScheduled(AbstractPutHDFSRecord.java:229)
      ... 16 common frames omitted

        Attachments

        1. Capture.PNG
          42 kB
          sujeeth

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sujeethg sujeeth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: