Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11092

hadoop shell commands should print usage if not given a class

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: scripts
    • Labels:
      None

      Description

      [root@bigtop-fedora-15 ~]# hdfs foobar
      Exception in thread "main" java.lang.NoClassDefFoundError: foobar
      Caused by: java.lang.ClassNotFoundException: foobar
      at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
      Could not find the main class: foobar. Program will exit.

      Instead of loading any class, it would be nice to explain the command is not valid and to call print_usage()

      1. HDFS-2565.patch
        2 kB
        Chu Tong
      2. HDFS-2565.patch
        2 kB
        Chu Tong
      3. HADOOP-11092.patch
        3 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          tomwhite Tom White added a comment -

          This is the same behaviour as other bin scripts, like 'hadoop' and 'mapred', so any fix should be for all of them.

          Show
          tomwhite Tom White added a comment - This is the same behaviour as other bin scripts, like 'hadoop' and 'mapred', so any fix should be for all of them.
          Hide
          stayhf Chu Tong added a comment -

          I tested this locally on my own dev cluster. No test is included as the changes are trivial.

          Show
          stayhf Chu Tong added a comment - I tested this locally on my own dev cluster. No test is included as the changes are trivial.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570046/HDFS-2565.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3987//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570046/HDFS-2565.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3987//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570053/HDFS-2565.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3988//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3988//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570053/HDFS-2565.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3988//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3988//console This message is automatically generated.
          Hide
          stayhf Chu Tong added a comment -

          since the changes are trivial, there is no test included and I tested this on my local development setup.

          Show
          stayhf Chu Tong added a comment - since the changes are trivial, there is no test included and I tested this on my local development setup.
          Hide
          aw Allen Wittenauer added a comment - - edited

          This breaks stuff.

          The big problem is that the patch assumes that every main method that one could call in the java code is documented in the shell code. This is definitely incorrect. In fact, the code has in the usage section:

             echo " or"
             echo "  CLASSNAME            run the class named CLASSNAME"
          

          In other words, that catchall at the end is meant to be a class. Thus why one throws a java error when one gives a broken subcommand.

          That said, what could be done here is that the code check for periods. If the catch-all doesn't contain a period, there is a 99% likelihood that it isn't a class but a typo.

          Show
          aw Allen Wittenauer added a comment - - edited This breaks stuff. The big problem is that the patch assumes that every main method that one could call in the java code is documented in the shell code. This is definitely incorrect. In fact, the code has in the usage section: echo " or" echo " CLASSNAME run the class named CLASSNAME" In other words, that catchall at the end is meant to be a class. Thus why one throws a java error when one gives a broken subcommand. That said, what could be done here is that the code check for periods. If the catch-all doesn't contain a period, there is a 99% likelihood that it isn't a class but a typo.
          Hide
          aw Allen Wittenauer added a comment -
          • Moving this to common
          • Rewriting the title
          Show
          aw Allen Wittenauer added a comment - Moving this to common Rewriting the title
          Hide
          aw Allen Wittenauer added a comment -
          • restores classname support to mapred
          • adds a new hadoop_validate_classname function
          • throws a usage message if the unknown subcommand doesn't contain a period (which also means we can safely remove the special handling for - )
          Show
          aw Allen Wittenauer added a comment - restores classname support to mapred adds a new hadoop_validate_classname function throws a usage message if the unknown subcommand doesn't contain a period (which also means we can safely remove the special handling for - )
          Hide
          sigje Jennifer Davis added a comment -

          +1

          Show
          sigje Jennifer Davis added a comment - +1
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1

          Show
          stevel@apache.org Steve Loughran added a comment - +1
          Hide
          aw Allen Wittenauer added a comment -

          Thanks!

          Committed to trunk.

          Show
          aw Allen Wittenauer added a comment - Thanks! Committed to trunk.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #690 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/690/)
          HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-mapreduce-project/bin/mapred
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #690 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/690/ ) HADOOP-11092 . hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062) hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/CHANGES.txt hadoop-mapreduce-project/bin/mapred hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1881 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1881/)
          HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-mapreduce-project/bin/mapred
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1881 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1881/ ) HADOOP-11092 . hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-mapreduce-project/bin/mapred hadoop-yarn-project/hadoop-yarn/bin/yarn
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1906 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1906/)
          HADOOP-11092. hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • hadoop-yarn-project/hadoop-yarn/bin/yarn
          • hadoop-mapreduce-project/bin/mapred
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1906 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1906/ ) HADOOP-11092 . hadoop shell commands should print usage if not given a class (aw) (aw: rev 3dc28e2052dd3a8e4cd5888fc4f9e7e37f8bc062) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs hadoop-common-project/hadoop-common/src/main/bin/hadoop hadoop-yarn-project/hadoop-yarn/bin/yarn hadoop-mapreduce-project/bin/mapred

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              bmahe Bruno Mahé
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development