Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3047

FileOutputCommitter throws wrong type of exception when calling abortTask() to handle a directory without permission

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      When FileOutputCommitter calls abortTask() to create a temp directory, if the user has no permission to access the directory, or a file with the same name has existed, of course it will fail, however the system will output the error information into the log file instead of throwing an exception.As a result, when the temp directory is needed later, since the temp directory hasn't been created yet, system will throw an exception to tell user that the temp directory doesn't exist.In my opinion, the exception is not exact and the error infomation will confuse users.

      1. MAPREDUCE-3047-2.patch
        0.9 kB
        madhukara phatak
      2. MAPREDUCE-3047-1.patch
        0.9 kB
        madhukara phatak
      3. MAPREDUCE-3047.patch
        2 kB
        JiangKai

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495197/MAPREDUCE-3047.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/797//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/797//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495197/MAPREDUCE-3047.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/797//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/797//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        This patch is out of date. the FileOutputCommitter has had major changes to it, and this patch needs to at least be upmerged for trunk

        Show
        Robert Joseph Evans added a comment - This patch is out of date. the FileOutputCommitter has had major changes to it, and this patch needs to at least be upmerged for trunk
        Hide
        madhukara phatak added a comment -

        Updated patch according to trunk

        Show
        madhukara phatak added a comment - Updated patch according to trunk
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522132/MAPREDUCE-3047-1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        -1 javac. The applied patch generated 508 javac compiler warnings (more than the trunk's current 507 warnings).

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService
        org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry
        org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization
        org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2186//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2186//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522132/MAPREDUCE-3047-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 508 javac compiler warnings (more than the trunk's current 507 warnings). +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.resourcemanager.TestClientRMService org.apache.hadoop.yarn.server.resourcemanager.resourcetracker.TestNMExpiry org.apache.hadoop.yarn.server.resourcemanager.TestAMAuthorization org.apache.hadoop.yarn.server.resourcemanager.TestApplicationACLs +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2186//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2186//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        I like the patch, I just have two comments before I can give it a +1.

        1. I need you to make sure you mark the contribute to apache link when submitting the patch. Without it I cannot take the patch.
        2. Our codeing convention calls for spaces instead of tabs.
        Show
        Robert Joseph Evans added a comment - I like the patch, I just have two comments before I can give it a +1. I need you to make sure you mark the contribute to apache link when submitting the patch. Without it I cannot take the patch. Our codeing convention calls for spaces instead of tabs.
        Hide
        madhukara phatak added a comment -

        updated the patch with space and granted the licence for inclusion

        Show
        madhukara phatak added a comment - updated the patch with space and granted the licence for inclusion
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12530013/MAPREDUCE-3047-2.patch
        against trunk revision 3241fc2.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5246//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530013/MAPREDUCE-3047-2.patch against trunk revision 3241fc2. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5246//console This message is automatically generated.

          People

          • Assignee:
            Unassigned
            Reporter:
            JiangKai
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development