Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.6.0
    • 2.7.0
    • task
    • Reviewed

    Description

      MAPREDUCE-5974 added in ability to instantiate multiple OCs, but if none of them are able to load it "throws" only a final a generic message: "Unable to initialize any output collector"

      The older behaviour was to throw the actual instantiation exception back, so it makes it to client logs with an actual meaningful error.

      Now the clients need to go take a look at the task's logs to find the WARNs that represent the failure in instantiation.

      Attachments

        Issue Links

        Activity

          qwertymaniac Harsh J added a comment -

          If someone would like to try working on this, the file to edit is hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java, and essentially we'd want some variable storing the last WARNed exception and rethrowing that at the end with a wrapped message such as "Final error in constructing the collector was:".

          A more clever impl. could include messages from each of the attempt exceptions, collecting them all, but the above would usually do for most of users only using the standard single/pair of collectors.

          qwertymaniac Harsh J added a comment - If someone would like to try working on this, the file to edit is hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java , and essentially we'd want some variable storing the last WARNed exception and rethrowing that at the end with a wrapped message such as "Final error in constructing the collector was:". A more clever impl. could include messages from each of the attempt exceptions, collecting them all, but the above would usually do for most of users only using the standard single/pair of collectors.
          varun_saxena Varun Saxena added a comment -

          Attaching a trivial patch. I think including messages from all the previous exceptions will make the output too verbose and difficult to understand.

          varun_saxena Varun Saxena added a comment - Attaching a trivial patch. I think including messages from all the previous exceptions will make the output too verbose and difficult to understand.
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12687152/MAPREDUCE-6194.patch
          against trunk revision 25a0440.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12687152/MAPREDUCE-6194.patch against trunk revision 25a0440. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 13 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5079//console This message is automatically generated.
          varun_saxena Varun Saxena added a comment -

          Findbugs unrelated

          varun_saxena Varun Saxena added a comment - Findbugs unrelated
          varun_saxena Varun Saxena added a comment -

          Harsh J, kindly review

          varun_saxena Varun Saxena added a comment - Harsh J , kindly review
          qwertymaniac Harsh J added a comment -

          Looks good to me, thanks very much Varun! Committing shortly.

          P.s. Please do not set a 'Fix Version', instead use the 'Target Version'. The former is used by the project to indicate which versions it has already been committed to, while the latter is for which version it needs to be committed to.

          qwertymaniac Harsh J added a comment - Looks good to me, thanks very much Varun! Committing shortly. P.s. Please do not set a 'Fix Version', instead use the 'Target Version'. The former is used by the project to indicate which versions it has already been committed to, while the latter is for which version it needs to be committed to.
          qwertymaniac Harsh J added a comment -

          Thanks again for your contribution Varun!

          Committed to trunk and branch-2.

          qwertymaniac Harsh J added a comment - Thanks again for your contribution Varun! Committed to trunk and branch-2.
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #6716 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6716/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #6716 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6716/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          varun_saxena Varun Saxena added a comment -

          Thanks for the review and commit Harsh J. Will use Target Version from next time onwards.

          varun_saxena Varun Saxena added a comment - Thanks for the review and commit Harsh J . Will use Target Version from next time onwards.
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #42 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/42/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #42 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/42/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #776 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/776/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #776 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/776/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1974 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1974/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          • hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1974 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1974/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #39 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/39/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          • hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #39 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/39/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #43 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/43/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          • hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #43 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/43/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java hadoop-mapreduce-project/CHANGES.txt
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1993 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1993/)
          MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1993 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1993/ ) MAPREDUCE-6194 . Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. (harsh: rev 298d09c9b583088f364038adcb1edf1eb1c2c196) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java

          People

            varun_saxena Varun Saxena Assign to me
            qwertymaniac Harsh J
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment

                  Client must be authenticated to access this resource.