Hadoop Common
  1. Hadoop Common
  2. HADOOP-8370

Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: native
    • Labels:
      None
    • Environment:

      Description

      [INFO] — native-maven-plugin:1.0-alpha-7:javah (default) @ hadoop-common —
      [INFO] /bin/sh -c cd /build/hadoop-common/hadoop-common-project/hadoop-common && /usr/lib/jvm/jdk1.7.0_02/bin/javah -d /build/hadoop-common/hadoop-common-project/hadoop-common/target/native/javah -classpath <...> org.apache.hadoop.io.compress.zlib.ZlibDecompressor org.apache.hadoop.security.JniBasedUnixGroupsMapping org.apache.hadoop.io.nativeio.NativeIO org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping org.apache.hadoop.io.compress.snappy.SnappyCompressor org.apache.hadoop.io.compress.snappy.SnappyDecompressor org.apache.hadoop.io.compress.lz4.Lz4Compressor org.apache.hadoop.io.compress.lz4.Lz4Decompressor org.apache.hadoop.util.NativeCrc32
      Cannot find annotation method 'value()' in type 'org.apache.hadoop.classification.InterfaceAudience.LimitedPrivate': class file for org.apache.hadoop.classification.InterfaceAudience not found
      Cannot find annotation method 'value()' in type 'org.apache.hadoop.classification.InterfaceAudience.LimitedPrivate'
      Error: cannot access org.apache.hadoop.classification.InterfaceStability
      class file for org.apache.hadoop.classification.InterfaceStability not found

      The fix for me was to changing the scope of hadoop-annotations from
      "provided" to "compile" in pom.xml:

      <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-annotations</artifactId>
      <scope>compile</scope>
      </dependency>

      For some reason, it was the only dependency with scope "provided".

      1. HADOOP-8370.patch
        0.5 kB
        Trevor Robinson

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12525950/HADOOP-8370.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/957//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/957//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525950/HADOOP-8370.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/957//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/957//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        +1 looks good to me. Seems like hadoop-annotations should be compile scope since it's not provided by the jdk.

        Show
        Eli Collins added a comment - +1 looks good to me. Seems like hadoop-annotations should be compile scope since it's not provided by the jdk.
        Hide
        Eli Collins added a comment -

        Rationale btw is that "provided This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime" [1] and since we do not "expect the JDK or a container to provide the dependency at runtime" it should not be provided
        because provided scope is not transitive it means eg the native build doesnt get the classes from hadoop-annotations.

        1. http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

        Perhaps though the native build should instead add an explicit dependency on hadoop-annotations?

        Show
        Eli Collins added a comment - Rationale btw is that "provided This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime" [1] and since we do not "expect the JDK or a container to provide the dependency at runtime" it should not be provided because provided scope is not transitive it means eg the native build doesnt get the classes from hadoop-annotations. 1. http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html Perhaps though the native build should instead add an explicit dependency on hadoop-annotations?
        Hide
        Eli Collins added a comment -

        Tucu mentioned offline that the main motivation for making this provided is making sure we don't end up with jdiff in the hadoop classpath, need to check that.

        Show
        Eli Collins added a comment - Tucu mentioned offline that the main motivation for making this provided is making sure we don't end up with jdiff in the hadoop classpath, need to check that.
        Hide
        Trevor Robinson added a comment -

        I'm not sure where to check for that. Does the classpath just contain all of the jars in the appropriate directory under share/hadoop?

        With this patch, jdiff-*.jar shows up in the same place as the official CDH4 build, only under share/hadoop/mapreduce/lib:

        $ find hadoop-dist/target/hadoop-2.0.0-cdh4.0.0 -name 'jdiff.jar'
        hadoop-dist/target/hadoop-2.0.0-cdh4.0.0/share/hadoop/mapreduce/lib/jdiff-1.0.9.jar

        Show
        Trevor Robinson added a comment - I'm not sure where to check for that. Does the classpath just contain all of the jars in the appropriate directory under share/hadoop? With this patch, jdiff-*.jar shows up in the same place as the official CDH4 build, only under share/hadoop/mapreduce/lib: $ find hadoop-dist/target/hadoop-2.0.0-cdh4.0.0 -name ' jdiff .jar' hadoop-dist/target/hadoop-2.0.0-cdh4.0.0/share/hadoop/mapreduce/lib/jdiff-1.0.9.jar
        Hide
        Trevor Robinson added a comment -

        Is there anything I can do to help resolve this issue?

        Also, any idea why it works for everyone else with scope 'provided'? Am I using a newer Maven version?

        Show
        Trevor Robinson added a comment - Is there anything I can do to help resolve this issue? Also, any idea why it works for everyone else with scope 'provided'? Am I using a newer Maven version?
        Hide
        Lars Hofhansl added a comment -

        I have the same problem, but only with JDK7. I see you're using JDK7 as well, maybe that's the culprit?

        Show
        Lars Hofhansl added a comment - I have the same problem, but only with JDK7. I see you're using JDK7 as well, maybe that's the culprit?
        Hide
        Eli Collins added a comment -

        It is, this only comes up with jdk7.

        Tucu, since Trevor confirmed that we don't end up with jdiff in the lib dir due to this change do you still object?

        Show
        Eli Collins added a comment - It is, this only comes up with jdk7. Tucu, since Trevor confirmed that we don't end up with jdiff in the lib dir due to this change do you still object?
        Hide
        Eli Collins added a comment -

        Btw I filed MAPREDUCE-4413 for the MR jdiff issue.

        Show
        Eli Collins added a comment - Btw I filed MAPREDUCE-4413 for the MR jdiff issue.
        Hide
        Thomas Graves added a comment -

        Can we commit this?

        Show
        Thomas Graves added a comment - Can we commit this?
        Hide
        Lars Hofhansl added a comment -

        +1 on commit (not that my votes counts, but anyway)

        Show
        Lars Hofhansl added a comment - +1 on commit (not that my votes counts, but anyway)
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Thomas Graves added a comment -

        Thanks Trevor! I've committed this to trunk, branch-2, and branch-0.23.

        Show
        Thomas Graves added a comment - Thanks Trevor! I've committed this to trunk, branch-2, and branch-0.23.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2606 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2606/)
        HADOOP-8370. Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2606 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2606/ ) HADOOP-8370 . Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2541 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2541/)
        HADOOP-8370. Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2541 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2541/ ) HADOOP-8370 . Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2561 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2561/)
        HADOOP-8370. Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2561 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2561/ ) HADOOP-8370 . Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #331 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/331/)
        merge -r 1367764:1367765 from branch-2. FIXES: HADOOP-8370 (Revision 1367766)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367766
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #331 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/331/ ) merge -r 1367764:1367765 from branch-2. FIXES: HADOOP-8370 (Revision 1367766) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367766 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1122 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1122/)
        HADOOP-8370. Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1122 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1122/ ) HADOOP-8370 . Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1154 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1154/)
        HADOOP-8370. Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1154 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1154/ ) HADOOP-8370 . Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found (Trevor Robinson via tgraves) (Revision 1367764) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1367764 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
        Hide
        Alejandro Abdelnur added a comment -

        Just realized we missed to do a exclusion in hadoop-client as part of this JIRA, opened a follow up one, HADOOP-8643.

        Show
        Alejandro Abdelnur added a comment - Just realized we missed to do a exclusion in hadoop-client as part of this JIRA, opened a follow up one, HADOOP-8643 .

          People

          • Assignee:
            Trevor Robinson
            Reporter:
            Trevor Robinson
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development