Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7325

hadoop command - do not accept class names starting with a hyphen

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.22.0
    • scripts
    • None
    • Reviewed

    Description

      If this is committed I will look at patches for hdfs and mapred.

      When teaching a good portion of the students in every single class execute:

      $ hadoop -fs ls /
      

      The -fs is passed directly to the JVM and the JVM fails to start:

      $ ./bin/hadoop -fs ls /
      Unrecognized option: -fs
      Could not create the Java virtual machine.
      

      Which is confusing and typically requires explanation. The attached patch improves that behavior:

      $ ./bin/hadoop -fs ls /
      Error: No command named `-fs' was found. Perhaps you meant `hadoop fs'
      

      The only risk I can see is if someone is abusing the implementation of hadoop command doing something like so:

      $ ./bin/hadoop -Xmx1g  org.apache.hadoop.util.RunJar
      RunJar jarFile [mainClass] args...
      

      The hadoop command does not appear to advertise allowing JVM options before the classname.

      Attachments

        1. hadoop-illegal-class-name-0.patch
          0.7 kB
          Brock Noland
        2. hadoop-illegal-class-name-1.patch
          0.5 kB
          Brock Noland
        3. hadoop-illegal-class-name-2.patch
          0.6 kB
          Brock Noland

        Issue Links

          Activity

            brocknoland Brock Noland added a comment -

            Attached implements requested improvement.

            brocknoland Brock Noland added a comment - Attached implements requested improvement.
            atm Aaron Myers added a comment -

            Hitting 'submit patch' so Hudson will run the pre-commit hooks.

            atm Aaron Myers added a comment - Hitting 'submit patch' so Hudson will run the pre-commit hooks.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12480193/hadoop-illegal-class-name-0.patch
            against trunk revision 1128385.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480193/hadoop-illegal-class-name-0.patch against trunk revision 1128385. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/535//console This message is automatically generated.
            tlipcon Todd Lipcon added a comment -

            nits:

            • I don't think you need the comments explaining bash [[ vs [
            • if you do want to keep them, there's a typo: bra*c*ket is missing its 'c'.
            tlipcon Todd Lipcon added a comment - nits: I don't think you need the comments explaining bash [[ vs [ if you do want to keep them, there's a typo: bra*c*ket is missing its 'c'.
            brocknoland Brock Noland added a comment -

            nit's resolved.

            brocknoland Brock Noland added a comment - nit's resolved.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12481405/hadoop-illegal-class-name-1.patch
            against trunk revision 1130833.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481405/hadoop-illegal-class-name-1.patch against trunk revision 1130833. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/572//console This message is automatically generated.
            brocknoland Brock Noland added a comment -

            Slightly too agressive on comment deletion. I think the first comment was useful. Updated.

            brocknoland Brock Noland added a comment - Slightly too agressive on comment deletion. I think the first comment was useful. Updated.
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12481408/hadoop-illegal-class-name-2.patch
            against trunk revision 1130833.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//testReport/
            Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481408/hadoop-illegal-class-name-2.patch against trunk revision 1130833. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/574//console This message is automatically generated.
            tlipcon Todd Lipcon added a comment -

            +1

            tlipcon Todd Lipcon added a comment - +1
            tlipcon Todd Lipcon added a comment -

            Committed to trunk and 0.22. Thanks, Brock! Please do file patches for bin/hdfs and bin/mapred in those projects as well.

            tlipcon Todd Lipcon added a comment - Committed to trunk and 0.22. Thanks, Brock! Please do file patches for bin/hdfs and bin/mapred in those projects as well.
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-trunk-Commit #643 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/643/)
            HADOOP-7325. The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland.

            todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132769
            Files :

            • /hadoop/common/trunk/bin/hadoop
            • /hadoop/common/trunk/CHANGES.txt
            hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #643 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/643/ ) HADOOP-7325 . The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132769 Files : /hadoop/common/trunk/bin/hadoop /hadoop/common/trunk/CHANGES.txt
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-22-branch #63 (See https://builds.apache.org/hudson/job/Hadoop-Common-22-branch/63/)
            HADOOP-7325. The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland.

            todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132768
            Files :

            • /hadoop/common/branches/branch-0.22/bin/hadoop
            • /hadoop/common/branches/branch-0.22/CHANGES.txt
            hudson Hudson added a comment - Integrated in Hadoop-Common-22-branch #63 (See https://builds.apache.org/hudson/job/Hadoop-Common-22-branch/63/ ) HADOOP-7325 . The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132768 Files : /hadoop/common/branches/branch-0.22/bin/hadoop /hadoop/common/branches/branch-0.22/CHANGES.txt
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-trunk #712 (See https://builds.apache.org/job/Hadoop-Common-trunk/712/)
            HADOOP-7325. The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland.

            todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132769
            Files :

            • /hadoop/common/trunk/bin/hadoop
            • /hadoop/common/trunk/CHANGES.txt
            hudson Hudson added a comment - Integrated in Hadoop-Common-trunk #712 (See https://builds.apache.org/job/Hadoop-Common-trunk/712/ ) HADOOP-7325 . The hadoop command should not accept class names starting with a hyphen. Contributed by Brock Noland. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1132769 Files : /hadoop/common/trunk/bin/hadoop /hadoop/common/trunk/CHANGES.txt
            brocknoland Brock Noland added a comment -

            Looking at the hdfs and mapred commands, they do not seem to allow the "command" to be a direct classname and also have unknown command handling built in, so I do not think they need to be updated. If someone disagrees, please share!

            elif [ "$COMMAND" = "groups" ] ; then
              CLASS=org.apache.hadoop.hdfs.tools.GetGroups
            else
              echo $COMMAND - invalid command
              print_usage
              exit
            
            brocknoland Brock Noland added a comment - Looking at the hdfs and mapred commands, they do not seem to allow the "command" to be a direct classname and also have unknown command handling built in, so I do not think they need to be updated. If someone disagrees, please share! elif [ "$COMMAND" = "groups" ] ; then CLASS=org.apache.hadoop.hdfs.tools.GetGroups else echo $COMMAND - invalid command print_usage exit
            tlipcon Todd Lipcon added a comment -

            Sounds right to me. Thanks for looking.

            tlipcon Todd Lipcon added a comment - Sounds right to me. Thanks for looking.

            People

              brocknoland Brock Noland
              brocknoland Brock Noland
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: