Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      There are a number of conversions in the Job History Event classes that are totally unnecessary. It appears that they were originally used to convert from the internal avro format, but now many of them do not pull the values from the avro they store them internally.

      For example:

      TaskAttemptFinishedEvent.java
        /** Get the task type */
        public TaskType getTaskType() {
          return TaskType.valueOf(taskType.toString());
        }
      

      The code currently is taking an enum, converting it to a string and then asking the same enum to convert it back to an enum. If java work properly this should be a noop and a reference to the original taskType should be returned.

      There are several places that a string is having toString called on it, and since strings are immutable it returns a reference to itself.

      The various ids are not immutable and probably should not be changed at this point.

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1337 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1337/)
        MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257)

        Result = SUCCESS
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1337 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1337/ ) MAPREDUCE-4822 . Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1309 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1309/)
        MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257)

        Result = FAILURE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1309 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1309/ ) MAPREDUCE-4822 . Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/)
        svn merge -c 1443257 FIXES: MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443271)

        Result = SUCCESS
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443271
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #518 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/518/ ) svn merge -c 1443257 FIXES: MAPREDUCE-4822 . Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443271) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443271 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Yarn-trunk #120 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/120/)
        MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257)

        Result = FAILURE
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Show
        Hudson added a comment - Integrated in Hadoop-Yarn-trunk #120 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/120/ ) MAPREDUCE-4822 . Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257) Result = FAILURE jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Hide
        Jason Lowe added a comment -

        Thanks, Chu! I committed this to trunk, branch-2, and branch-0.23.

        Note that for future JIRAs, the Fix Version field should only be set when the fix is finally committed. The Target Version field should be used when indicating what version(s) a patch is targeting before it is committed.

        Show
        Jason Lowe added a comment - Thanks, Chu! I committed this to trunk, branch-2, and branch-0.23. Note that for future JIRAs, the Fix Version field should only be set when the fix is finally committed. The Target Version field should be used when indicating what version(s) a patch is targeting before it is committed.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-trunk-Commit #3335 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3335/)
        MAPREDUCE-4822. Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257)

        Result = SUCCESS
        jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Show
        Hudson added a comment - Integrated in Hadoop-trunk-Commit #3335 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3335/ ) MAPREDUCE-4822 . Unnecessary conversions in History Events. Contributed by Chu Tong (Revision 1443257) Result = SUCCESS jlowe : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1443257 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/MapAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/ReduceAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskAttemptFinishedEvent.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/TaskFinishedEvent.java
        Hide
        Chu Tong added a comment -

        right ..., Thanks Jason!

        Show
        Chu Tong added a comment - right ..., Thanks Jason!
        Hide
        Jason Lowe added a comment -

        +1, lgtm. There's one minor nit where we're now needlessly checking for null:

             if(successfulAttemptId != null)
             {
               return successfulAttemptId;
             }
             return null;
        

        but I'll just fix that as part of the commit.

        Show
        Jason Lowe added a comment - +1, lgtm. There's one minor nit where we're now needlessly checking for null: if (successfulAttemptId != null ) { return successfulAttemptId; } return null ; but I'll just fix that as part of the commit.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12568300/MAPREDUCE-4822.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3312//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3312//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12568300/MAPREDUCE-4822.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3312//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3312//console This message is automatically generated.
        Hide
        Chu Tong added a comment -

        Ok, I think your point is more solid. I made the changes as you instructed. Thank you.

        Show
        Chu Tong added a comment - Ok, I think your point is more solid. I made the changes as you instructed. Thank you.
        Hide
        Jason Lowe added a comment -

        The intent of TaskAttemptID.forName() is to be the inverse of its toString(). In these cases we have a valid TaskAttemptID, so the IllegalArgumentException should never be thrown in this context unless forName() or toString() is broken. And I can't imagine why someone would expect and rely on that behavior if it somehow were throwing it. We have an ID, someone is asking for it, why convert it to a string and back before returning it?

        To sum up, return TaskAttemptID.forName(taskAttempt.toString()) should just be return taskAttempt. If you're not comfortable making the change, that's fine let me know. We can either file a followup JIRA or I can post the changes to the patch.

        Show
        Jason Lowe added a comment - The intent of TaskAttemptID.forName() is to be the inverse of its toString(). In these cases we have a valid TaskAttemptID, so the IllegalArgumentException should never be thrown in this context unless forName() or toString() is broken. And I can't imagine why someone would expect and rely on that behavior if it somehow were throwing it. We have an ID, someone is asking for it, why convert it to a string and back before returning it? To sum up, return TaskAttemptID.forName(taskAttempt.toString()) should just be return taskAttempt . If you're not comfortable making the change, that's fine let me know. We can either file a followup JIRA or I can post the changes to the patch.
        Hide
        Chu Tong added a comment -

        My primary concern is the error checking to throw the exception at the end of the function and I do not know if we want to get ride of that.

        Show
        Chu Tong added a comment - My primary concern is the error checking to throw the exception at the end of the function and I do not know if we want to get ride of that.
        Hide
        Jason Lowe added a comment -

        That code simply parses a string into a TaskAttemptID, the very thing we're trying to avoid doing unnecessarily when we already have the TaskAttemptID. Is there something specific about that conversion you're concerned is necessary when passing an existing TaskAttemptID as a string?

        Show
        Jason Lowe added a comment - That code simply parses a string into a TaskAttemptID, the very thing we're trying to avoid doing unnecessarily when we already have the TaskAttemptID. Is there something specific about that conversion you're concerned is necessary when passing an existing TaskAttemptID as a string?
        Hide
        Chu Tong added a comment -

        Sorry, I did not put the code in codeblock style.

        Show
        Chu Tong added a comment - Sorry, I did not put the code in codeblock style.
        Hide
        Chu Tong added a comment -

        When I look at the code the TaskAttemptID.forName method does not look like a simple conversion. Therefore, I think it is safer to leave what it is right now.

        public static TaskAttemptID forName(String str
        ) throws IllegalArgumentException {
        if(str == null)
        return null;
        String exceptionMsg = null;
        try {
        String[] parts = str.split(Character.toString(SEPARATOR));
        if(parts.length == 6) {
        if(parts[0].equals(ATTEMPT)) {
        String type = parts[3];
        TaskType t = TaskID.getTaskType(type.charAt(0));
        if(t != null)

        { return new org.apache.hadoop.mapred.TaskAttemptID (parts[1], Integer.parseInt(parts[2]), t, Integer.parseInt(parts[4]), Integer.parseInt(parts[5])); }

        else
        exceptionMsg = "Bad TaskType identifier. TaskAttemptId string : "
        + str + " is not properly formed.";
        }
        }
        } catch (Exception ex)

        { //fall below }

        if (exceptionMsg == null)

        { exceptionMsg = "TaskAttemptId string : " + str + " is not properly formed"; }

        throw new IllegalArgumentException(exceptionMsg);
        }

        Show
        Chu Tong added a comment - When I look at the code the TaskAttemptID.forName method does not look like a simple conversion. Therefore, I think it is safer to leave what it is right now. public static TaskAttemptID forName(String str ) throws IllegalArgumentException { if(str == null) return null; String exceptionMsg = null; try { String[] parts = str.split(Character.toString(SEPARATOR)); if(parts.length == 6) { if(parts [0] .equals(ATTEMPT)) { String type = parts [3] ; TaskType t = TaskID.getTaskType(type.charAt(0)); if(t != null) { return new org.apache.hadoop.mapred.TaskAttemptID (parts[1], Integer.parseInt(parts[2]), t, Integer.parseInt(parts[4]), Integer.parseInt(parts[5])); } else exceptionMsg = "Bad TaskType identifier. TaskAttemptId string : " + str + " is not properly formed."; } } } catch (Exception ex) { //fall below } if (exceptionMsg == null) { exceptionMsg = "TaskAttemptId string : " + str + " is not properly formed"; } throw new IllegalArgumentException(exceptionMsg); }
        Hide
        Jason Lowe added a comment -

        I see there's another unnecessary conversion category that we've missed, and that's the TaskIDs and TaskAttemptIDs. They look like the following:

          public TaskAttemptID getAttemptId() {
            return TaskAttemptID.forName(attemptId.toString());
          }
        ..
          /** Get task id */
          public TaskID getTaskId() { return TaskID.forName(taskid.toString()); }
          /** Get successful task attempt id */
          public TaskAttemptID getSuccessfulTaskAttemptId() {
            if(successfulAttemptId != null)
            {
              return TaskAttemptID.forName(successfulAttemptId.toString());
            }
            return null;
          }
        

        Sorry I didn't catch this previously.

        Show
        Jason Lowe added a comment - I see there's another unnecessary conversion category that we've missed, and that's the TaskIDs and TaskAttemptIDs. They look like the following: public TaskAttemptID getAttemptId() { return TaskAttemptID.forName(attemptId.toString()); } .. /** Get task id */ public TaskID getTaskId() { return TaskID.forName(taskid.toString()); } /** Get successful task attempt id */ public TaskAttemptID getSuccessfulTaskAttemptId() { if (successfulAttemptId != null ) { return TaskAttemptID.forName(successfulAttemptId.toString()); } return null ; } Sorry I didn't catch this previously.
        Hide
        Chu Tong added a comment -

        No testcase is included as the change is trivial.

        Show
        Chu Tong added a comment - No testcase is included as the change is trivial.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12568110/MAPREDUCE-4822.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3306//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3306//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12568110/MAPREDUCE-4822.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3306//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3306//console This message is automatically generated.
        Hide
        Chu Tong added a comment -

        Changed all affected files under jobhistory directory

        Show
        Chu Tong added a comment - Changed all affected files under jobhistory directory
        Hide
        Jason Lowe added a comment -

        Hi Chu, thanks for picking this up.

        The original intent of the JIRA was to fix more than just the one example occurrence that Bobby pointed out. Looking through the source, I can see more occurrences of unnecessary conversions in MapAttemptFinishedEvent and TaskFinishedEvent.

        There's a number of other event classes, like TaskStartedEvent, that are converting back and forth between strings and enums since they store their internal state in a single datum Avro record. We can refactor these to build the datum on-demand like TaskAttemptFinishedEvent does, but that's probably best left to another JIRA. We can keep this one short and simple.

        Show
        Jason Lowe added a comment - Hi Chu, thanks for picking this up. The original intent of the JIRA was to fix more than just the one example occurrence that Bobby pointed out. Looking through the source, I can see more occurrences of unnecessary conversions in MapAttemptFinishedEvent and TaskFinishedEvent. There's a number of other event classes, like TaskStartedEvent, that are converting back and forth between strings and enums since they store their internal state in a single datum Avro record. We can refactor these to build the datum on-demand like TaskAttemptFinishedEvent does, but that's probably best left to another JIRA. We can keep this one short and simple.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12567961/MAPREDUCE-4822.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3302//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3302//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12567961/MAPREDUCE-4822.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3302//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3302//console This message is automatically generated.
        Hide
        Chu Tong added a comment -

        No testcase is included as the change is trivial.

        For JavaDoc warnings, it is false positive as the same number of warnings are generated on a clean build under my dev environment.

        -1 overall.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated 20 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        Show
        Chu Tong added a comment - No testcase is included as the change is trivial. For JavaDoc warnings, it is false positive as the same number of warnings are generated on a clean build under my dev environment. -1 overall. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated 20 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version ) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12567924/MAPREDUCE-4822.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3299//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12567924/MAPREDUCE-4822.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3299//console This message is automatically generated.
        Hide
        Chu Tong added a comment -

        Removed unnecessary conversion, please review.

        Show
        Chu Tong added a comment - Removed unnecessary conversion, please review.

          People

          • Assignee:
            Chu Tong
            Reporter:
            Robert Joseph Evans
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development