Hadoop Common
  1. Hadoop Common
  2. HADOOP-8623

hadoop jar command should respect HADOOP_OPTS

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: 3.0.0, 2.0.2-alpha, 0.23.4
    • Component/s: scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The jar command to the hadoop script should use any set HADOOP_OPTS and HADOOP_CLIENT_OPTS environment variables like all the other commands.

      1. HADOOP-8623.patch
        2 kB
        Steven Willis

        Activity

        Hide
        Steven Willis added a comment -

        Like HADOOP-7491 which added these environment variables to the case where the hadoop script is run with a classname, they should also be used when run with jar as the command. All the other commands use these env vars, I can't see why jar doesn't.

        Show
        Steven Willis added a comment - Like HADOOP-7491 which added these environment variables to the case where the hadoop script is run with a classname, they should also be used when run with jar as the command. All the other commands use these env vars, I can't see why jar doesn't.
        Hide
        Steven Willis added a comment -

        Since every case in the big if block have would now had:

        HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"

        I just removed each individual line and added a single line below the block. It has the nice side affect of making that big block more readable

        Show
        Steven Willis added a comment - Since every case in the big if block have would now had: HADOOP_OPTS= "$HADOOP_OPTS $HADOOP_CLIENT_OPTS" I just removed each individual line and added a single line below the block. It has the nice side affect of making that big block more readable
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537914/HADOOP-8623.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.ha.TestZKFailoverController

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1220//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1220//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537914/HADOOP-8623.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1220//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1220//console This message is automatically generated.
        Hide
        Arpit Gupta added a comment -

        the same is being repeated in hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs and hadoop-mapreduce-project/bin/mapred scripts. I think we can move assignment of HADOOP_OPTS to before the case statement and remove the line from various if statements in bin/mapred and bin/hdfs.

        Show
        Arpit Gupta added a comment - the same is being repeated in hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs and hadoop-mapreduce-project/bin/mapred scripts. I think we can move assignment of HADOOP_OPTS to before the case statement and remove the line from various if statements in bin/mapred and bin/hdfs.
        Hide
        Arpit Gupta added a comment -

        Actually thinking about it more, hdfs and mapred could have separate configs and can be invoked independently from the bin/hadoop script so the current changes look good to go.

        +1

        Show
        Arpit Gupta added a comment - Actually thinking about it more, hdfs and mapred could have separate configs and can be invoked independently from the bin/hadoop script so the current changes look good to go. +1
        Hide
        Daryn Sharp added a comment -

        +1 ditto

        Show
        Daryn Sharp added a comment - +1 ditto
        Hide
        Suresh Srinivas added a comment -

        Thanks for contributing the patch Steven. I have added you as a contributor to Hadoop common. You can now assign the jiras to yourself.

        I committed the patch to trunk. Will merge it into 2.x next.

        Show
        Suresh Srinivas added a comment - Thanks for contributing the patch Steven. I have added you as a contributor to Hadoop common. You can now assign the jiras to yourself. I committed the patch to trunk. Will merge it into 2.x next.
        Hide
        Jason Lowe added a comment -

        I merged this to branch-0.23 as well.

        Show
        Jason Lowe added a comment - I merged this to branch-0.23 as well.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #388 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/388/)
        svn merge -c 1366126 FIXES: HADOOP-8623. hadoop jar command should respect HADOOP_OPTS. Contributed by Steven Willis. (Revision 1391276)

        Result = UNSTABLE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1391276
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #388 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/388/ ) svn merge -c 1366126 FIXES: HADOOP-8623 . hadoop jar command should respect HADOOP_OPTS. Contributed by Steven Willis. (Revision 1391276) Result = UNSTABLE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1391276 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/bin/hadoop
        Hide
        Steven Willis added a comment -

        I am out of the office until Tuesday October 16th. Please contact Seth Madison for any urgent issues.

        Show
        Steven Willis added a comment - I am out of the office until Tuesday October 16th. Please contact Seth Madison for any urgent issues.

          People

          • Assignee:
            Steven Willis
            Reporter:
            Steven Willis
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development