Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5980

filter empty string in the libjars when submitting jobs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      -libjars is used to upload jars into hdfs when submitting jobs. If there is empty string between two comma, the copying processing will try to copy all the subdirs/files under current directory. I believe it is not user's expected behavior.

        Activity

        Hide
        zhihai xu added a comment -

        the other JIRA HADOOP-10820 address the same issue as this one.
        https://issues.apache.org/jira/browse/HADOOP-10820
        :Empty entry in libjars results in working directory being recursively localized

        Show
        zhihai xu added a comment - the other JIRA HADOOP-10820 address the same issue as this one. https://issues.apache.org/jira/browse/HADOOP-10820 :Empty entry in libjars results in working directory being recursively localized
        Hide
        Bing Jiang added a comment -

        I have noticed that org.apache.hadoop.util.GenericOptionsParser has a function validateFiles to validate the value of -libjars. So if it pass the value through -libjars, it will convert the empty into current working diectory path, then it pass the value via conf.get("tmpjars"). So my first patch cannot solve the issue. Need to think it over.

        Show
        Bing Jiang added a comment - I have noticed that org.apache.hadoop.util.GenericOptionsParser has a function validateFiles to validate the value of -libjars. So if it pass the value through -libjars, it will convert the empty into current working diectory path, then it pass the value via conf.get("tmpjars"). So my first patch cannot solve the issue. Need to think it over.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12656476/MAPREDUCE-5980.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4755//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4755//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12656476/MAPREDUCE-5980.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4755//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4755//console This message is automatically generated.

          People

          • Assignee:
            Bing Jiang
            Reporter:
            Bing Jiang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development