Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1863

[Rumen] Null failedMapAttemptCDFs in job traces generated by Rumen

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: tools/rumen
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      All the traces generated by Rumen for jobs having failed task attempts has null value for failedMapAttemptCDFs.

      1. rumen-npe-v1.1.patch
        1 kB
        Amar Kamat
      2. counters-test-trace.json.gz
        3 kB
        Amar Kamat
      3. dispatch-trace-output.json.gz
        27 kB
        Amar Kamat
      4. rumen-npe-v1.1-bin.patch
        62 kB
        Amar Kamat
      5. mr-1863-yhadoop-20.1xx.patch
        62 kB
        Hong Tang

        Activity

        Hide
        amar_kamat Amar Kamat added a comment -

        Attaching a patch that fixes the NPE. test-patch and ant-tests passed on my box. Note that this patch has resulted into changes to the following files

        src/test/tools/data/rumen/small-trace-test/dispatch-trace-output.json.gz
        src/test/tools/data/rumen/small-trace-test/counters-test-trace.json.gz
        

        Attached the same. The changes are that failedMapAttemptCDFs in the earlier expected-output files were null which is no longer the case.

        Show
        amar_kamat Amar Kamat added a comment - Attaching a patch that fixes the NPE. test-patch and ant-tests passed on my box. Note that this patch has resulted into changes to the following files src/test/tools/data/rumen/small-trace-test/dispatch-trace-output.json.gz src/test/tools/data/rumen/small-trace-test/counters-test-trace.json.gz Attached the same. The changes are that failedMapAttemptCDFs in the earlier expected-output files were null which is no longer the case.
        Hide
        amar_kamat Amar Kamat added a comment -

        Reattaching the expected output files.

        Show
        amar_kamat Amar Kamat added a comment - Reattaching the expected output files.
        Hide
        dking Dick King added a comment -

        The code change looks right.

        Please add the binary files to the patch, which you need to do to get the Hudson job to work correctly.

        Show
        dking Dick King added a comment - The code change looks right. Please add the binary files to the patch, which you need to do to get the Hudson job to work correctly.
        Hide
        amar_kamat Amar Kamat added a comment -

        Attaching a patch that includes the binary diffs of the gold-standard files.

        Show
        amar_kamat Amar Kamat added a comment - Attaching a patch that includes the binary diffs of the gold-standard files.
        Hide
        amar_kamat Amar Kamat added a comment -

        Running through hudson.

        Show
        amar_kamat Amar Kamat added a comment - Running through hudson.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448019/rumen-npe-v1.1-bin.patch
        against trunk revision 957772.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 9 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448019/rumen-npe-v1.1-bin.patch against trunk revision 957772. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/268/console This message is automatically generated.
        Hide
        amar_kamat Amar Kamat added a comment -

        The contrib test failure is because of TestSimulatorDeterministicReplay timeout. This is a known issue, see MAPREDUCE-1834.

        Show
        amar_kamat Amar Kamat added a comment - The contrib test failure is because of TestSimulatorDeterministicReplay timeout. This is a known issue, see MAPREDUCE-1834 .
        Hide
        chris.douglas Chris Douglas added a comment -

        +1

        I committed this. Thanks, Amar!

        Show
        chris.douglas Chris Douglas added a comment - +1 I committed this. Thanks, Amar!
        Hide
        hong.tang Hong Tang added a comment -

        patch for yahoop hadoop 20.1xx branch. Not to be committed.

        Show
        hong.tang Hong Tang added a comment - patch for yahoop hadoop 20.1xx branch. Not to be committed.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )

          People

          • Assignee:
            amar_kamat Amar Kamat
            Reporter:
            amar_kamat Amar Kamat
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development