Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11150

hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: scripts
    • Labels:
      None

      Description

      hadoop_validate_classname checks whether the classname contains ".". It is possible that classname without package is used in some examples or tutorials.

      1. HADOOP-11150-0.patch
        0.5 kB
        Masatake Iwasaki
      2. HADOOP-11150-1.patch
        0.7 kB
        Masatake Iwasaki

        Issue Links

          Activity

          Hide
          iwasakims Masatake Iwasaki added a comment -

          This patch changes the check condition. I do not think strict check for class name is necessary here.
          http://docs.oracle.com/javase/specs/jls/se7/html/jls-3.html#jls-3.8

          Show
          iwasakims Masatake Iwasaki added a comment - This patch changes the check condition. I do not think strict check for class name is necessary here. http://docs.oracle.com/javase/specs/jls/se7/html/jls-3.html#jls-3.8
          Hide
          aw Allen Wittenauer added a comment -

          This is an incomplete fix, as this now causes things like "hadoop -jar" or "hadoop thisisnotacommand" to generate a stack trace instead of usage, defeating the purpose of the original code. See HADOOP-11092.

          It should be noted that, AFAIK, all of the code that ships with Hadoop is in a package. The current code breaks exactly one situation: someone has injected Hadoop's classpath with their own jars where the main method is not in a package. While it's certainly possible, it's also unlikely in the vast majority of cases and probably not worth the much larger frustration of going through the entire java launch sequence just to be told you typo'd.

          That said, HADOOP-11092 should probably be marked as an incompatible change with a release note that basically says if you are doing the above situation, sorry, but it won't work anymore; stick it in a package.

          Show
          aw Allen Wittenauer added a comment - This is an incomplete fix, as this now causes things like "hadoop -jar" or "hadoop thisisnotacommand" to generate a stack trace instead of usage, defeating the purpose of the original code. See HADOOP-11092 . It should be noted that, AFAIK, all of the code that ships with Hadoop is in a package. The current code breaks exactly one situation: someone has injected Hadoop's classpath with their own jars where the main method is not in a package. While it's certainly possible, it's also unlikely in the vast majority of cases and probably not worth the much larger frustration of going through the entire java launch sequence just to be told you typo'd. That said, HADOOP-11092 should probably be marked as an incompatible change with a release note that basically says if you are doing the above situation, sorry, but it won't work anymore; stick it in a package.
          Hide
          iwasakims Masatake Iwasaki added a comment -

          Thanks for the comment Allen Wittenauer. I misunderstood that the change came from HADOOP-9902. I assumed the situation like trivial experiment by writing a few lines of code and javac'ing it and run.
          If this is the spec, how about showing the message like "The first argument is not valid command nor fully qualified class name" before usage? Otherwise, is it worth to add command such as "run" which takes main class name as argument?

          Show
          iwasakims Masatake Iwasaki added a comment - Thanks for the comment Allen Wittenauer . I misunderstood that the change came from HADOOP-9902 . I assumed the situation like trivial experiment by writing a few lines of code and javac'ing it and run. If this is the spec, how about showing the message like "The first argument is not valid command nor fully qualified class name" before usage? Otherwise, is it worth to add command such as "run" which takes main class name as argument?
          Hide
          aw Allen Wittenauer added a comment -

          how about showing the message like "The first argument is not valid command nor fully qualified class name" before usage?

          Yes, this is a good idea and really it should have been done as part of 11092. hadoop_validate_classname really should throw a hadoop_error message so that the user knows why they are getting usage. It'd also be good to have some comments around _validate to explain why that function exists.

          Otherwise, is it worth to add command such as "run" which takes main class name as argument?

          It's such an edge case that I don't think it is, especially if we call out in the release notes/documentation that classname needs to be in a package. I have to check, but I suspect this functionality is mostly undocumented anyway though.

          Show
          aw Allen Wittenauer added a comment - how about showing the message like "The first argument is not valid command nor fully qualified class name" before usage? Yes, this is a good idea and really it should have been done as part of 11092. hadoop_validate_classname really should throw a hadoop_error message so that the user knows why they are getting usage. It'd also be good to have some comments around _validate to explain why that function exists. Otherwise, is it worth to add command such as "run" which takes main class name as argument? It's such an edge case that I don't think it is, especially if we call out in the release notes/documentation that classname needs to be in a package. I have to check, but I suspect this functionality is mostly undocumented anyway though.
          Hide
          iwasakims Masatake Iwasaki added a comment -

          attaching updated patch which adds comments and error message on failure.

          Show
          iwasakims Masatake Iwasaki added a comment - attaching updated patch which adds comments and error message on failure.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12671920/HADOOP-11150-1.patch
          against trunk revision 0577eb3.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.crypto.random.TestOsSecureRandom

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4834//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4834//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12671920/HADOOP-11150-1.patch against trunk revision 0577eb3. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.crypto.random.TestOsSecureRandom +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4834//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4834//console This message is automatically generated.
          Hide
          iwasakims Masatake Iwasaki added a comment -

          The test failure seems to be irrelevant to this patch. I added no unit test because this change is just adding of trivial logging and I tested it by hand on my env.

          Show
          iwasakims Masatake Iwasaki added a comment - The test failure seems to be irrelevant to this patch. I added no unit test because this change is just adding of trivial logging and I tested it by hand on my env.
          Hide
          aw Allen Wittenauer added a comment -

          +1 will commit to trunk.

          Thanks!

          Show
          aw Allen Wittenauer added a comment - +1 will commit to trunk. Thanks!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #6537 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6537/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6537 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6537/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/5/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/5/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #743 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/743/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #743 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/743/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1933 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1933/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1933 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1933/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/5/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/5/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1957 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1957/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1957 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1957/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/5/)
          HADOOP-11150. hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #5 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/5/ ) HADOOP-11150 . hadoop command should show the reason on failure by invalid COMMAND or CLASSNAME (Masatake Iwasaki via aw) (aw: rev aee68b67f3894e19757f6e4f3ffb6ca10b9a0d64) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh

            People

            • Assignee:
              iwasakims Masatake Iwasaki
              Reporter:
              iwasakims Masatake Iwasaki
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development