Hadoop Common
  1. Hadoop Common
  2. HADOOP-7999

"hadoop archive" fails with ClassNotFoundException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: scripts
    • Labels:
      None

      Description

      Running "hadoop archive" from a command prompt results in this error:

      Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/tools/HadoopArchives
      Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.tools.HadoopArchives
      at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
      Could not find the main class: org.apache.hadoop.tools.HadoopArchives. Program will exit.

      The hadoop front-end script expects the TOOL_PATH environment variable to be set, but nothing provides a default value for it if it is not set. Since $TOOL_PATH expands to nothing, the hadoop-archives jar under share/hadoop/tools/lib isn't found, and we end up with a ClassNotFound exception.

        Activity

        Hide
        Jason Lowe added a comment -

        Patch updates hadoop-config.sh to set a default TOOL_PATH if none provided. No tests since it's a shell script change. Manually verified that an archive could be created via "hadoop archive" with this patch.

        Show
        Jason Lowe added a comment - Patch updates hadoop-config.sh to set a default TOOL_PATH if none provided. No tests since it's a shell script change. Manually verified that an archive could be created via "hadoop archive" with this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12512237/HADOOP-7999.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated 7 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/543//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/543//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512237/HADOOP-7999.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 7 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/543//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/543//console This message is automatically generated.
        Hide
        John George added a comment -

        +1
        Downloaded the patch and ran the following:

        hadoop archive
        2012-01-27 16:31:50,621 WARN conf.Configuration (Configuration.java:set(629)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
        archive -archiveName NAME -p <parent path> <src>* <dest>

        Invalid usage.

        and then ran
        $ hadoop org.apache.hadoop.tools.HadoopArchives
        Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/tools/HadoopArchives

        Both behavior as expected. That is: HadoopArchives in hadoop path only if 'archive' is given as a sub command.

        Show
        John George added a comment - +1 Downloaded the patch and ran the following: hadoop archive 2012-01-27 16:31:50,621 WARN conf.Configuration (Configuration.java:set(629)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used archive -archiveName NAME -p <parent path> <src>* <dest> Invalid usage. and then ran $ hadoop org.apache.hadoop.tools.HadoopArchives Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/tools/HadoopArchives Both behavior as expected. That is: HadoopArchives in hadoop path only if 'archive' is given as a sub command.
        Hide
        Mahadev konar added a comment -

        John,
        Thanks for verifying. Lessens my work . Can you please verify if a archive job runs fully?

        Show
        Mahadev konar added a comment - John, Thanks for verifying. Lessens my work . Can you please verify if a archive job runs fully?
        Hide
        John George added a comment -

        Tested that I could create an archive, list an archive, run a wordcount on a file in the archive and the tests passed.

        $ hadoop archive -archiveName tmp.har -p /user/JG tmp /user/JG/

        $ hadoop fs -ls har://user/johngeo/tmp.har/tmp

        $hadoop jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar wordcount har:///user/JG/tmp.har/tmp/newPatchFindbugsWarnings.html output111

        Show
        John George added a comment - Tested that I could create an archive, list an archive, run a wordcount on a file in the archive and the tests passed. $ hadoop archive -archiveName tmp.har -p /user/JG tmp /user/JG/ $ hadoop fs -ls har://user/johngeo/tmp.har/tmp $hadoop jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar wordcount har:///user/JG/tmp.har/tmp/newPatchFindbugsWarnings.html output111
        Hide
        Mahadev konar added a comment -

        Thanks John. Ill go ahead and commit this now!

        Show
        Mahadev konar added a comment - Thanks John. Ill go ahead and commit this now!
        Hide
        Mahadev konar added a comment -

        I just committed this to trunk and 0.23. Thanks Jason and John.

        Show
        Mahadev konar added a comment - I just committed this to trunk and 0.23. Thanks Jason and John.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1689 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1689/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1689 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1689/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #446 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/446/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk.

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #446 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/446/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #437 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/437/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk.

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #437 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/437/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1617 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1617/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1617 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1617/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1634 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1634/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1634 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1634/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #461 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/461/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk.

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #461 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/461/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #942 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/942/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #942 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/942/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #155 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/155/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk.

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #155 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/155/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/177/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk.

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/177/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) - Merging r1237915 from trunk. mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237916 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #975 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/975/)
        HADOOP-7999. "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev)

        mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #975 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/975/ ) HADOOP-7999 . "hadoop archive" fails with ClassNotFoundException. (Jason Lowe via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1237915 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.sh

          People

          • Assignee:
            Jason Lowe
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development