Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1915

IndexCache - getIndexInformation - check reduce index Out Of Bounds

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When checking if the "reduce" index is out of bounds the check should be

      info.mapSpillRecord.size() <= reduce

      instead of:

      info.mapSpillRecord.size() < reduce

      Not a big bug since an Out Of Bounds is thrown downstream anyway.

      1. MAPREDUCE-1915_1.patch
        3 kB
        Priyo Mustafi
      2. MAPREDUCE-1915.patch
        0.6 kB
        Rares Vernica

        Activity

        Rares Vernica created issue -
        Rares Vernica made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-1915.patch [ 12448730 ]
        Rares Vernica made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Affects Version/s 0.20.1 [ 12314047 ]
        Affects Version/s 0.20.3 [ 12314813 ]
        Affects Version/s 0.21.0 [ 12314045 ]
        Affects Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.20.3 [ 12314813 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.20.2 [ 12314205 ]
        Fix Version/s 0.20.1 [ 12314047 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448730/MAPREDUCE-1915.patch
        against trunk revision 960534.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448730/MAPREDUCE-1915.patch against trunk revision 960534. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/console This message is automatically generated.
        Tom White made changes -
        Fix Version/s 0.20.2 [ 12314205 ]
        Fix Version/s 0.21.1 [ 12315272 ]
        Fix Version/s 0.21.0 [ 12314045 ]
        Fix Version/s 0.20.1 [ 12314047 ]
        Nigel Daley made changes -
        Fix Version/s 0.22.0 [ 12314184 ]
        Fix Version/s 0.20.3 [ 12314813 ]
        Fix Version/s 0.21.1 [ 12315272 ]
        Hide
        Priyo Mustafi added a comment -

        Hi Rares,
        The fix makes sense. I am writing a testcase for it but then I am not sure when this scenario is going to ever occur. Looks like you noticed an OutOfBoundException but under what circumstance?

        Show
        Priyo Mustafi added a comment - Hi Rares, The fix makes sense. I am writing a testcase for it but then I am not sure when this scenario is going to ever occur. Looks like you noticed an OutOfBoundException but under what circumstance?
        Hide
        Priyo Mustafi added a comment -

        Added testcase. The latest attachment contains both the fix and the testcase.

        Show
        Priyo Mustafi added a comment - Added testcase. The latest attachment contains both the fix and the testcase.
        Priyo Mustafi made changes -
        Attachment MAPREDUCE-1915_1.patch [ 12469489 ]
        Hide
        Konstantin Shvachko added a comment -

        +1 I liked the test case. Please assign it to 0.22 as a bug fix.

        Show
        Konstantin Shvachko added a comment - +1 I liked the test case. Please assign it to 0.22 as a bug fix.
        Priyo Mustafi made changes -
        Assignee Priyo Mustafi [ priyomustafi ]
        Priyo Mustafi made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Priyo Mustafi added a comment -

        Added to 0.22

        Show
        Priyo Mustafi added a comment - Added to 0.22
        Priyo Mustafi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.22.0 [ 12314184 ]
        Hide
        Priyo Mustafi added a comment -

        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] -1 release audit. The applied patch generated 20 release audit warnings (more than the trunk's current 1 warnings).
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.

        Show
        Priyo Mustafi added a comment - [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 release audit. The applied patch generated 20 release audit warnings (more than the trunk's current 1 warnings). [exec] [exec] +1 system test framework. The patch passed system test framework compile.
        Hide
        Priyo Mustafi added a comment -

        Got a +1 on release audit as well. It was an environmental issue. Patch-test and test all passed now.

        Show
        Priyo Mustafi added a comment - Got a +1 on release audit as well. It was an environmental issue. Patch-test and test all passed now.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this. Thank you Priyo.

        Show
        Konstantin Shvachko added a comment - I just committed this. Thank you Priyo.
        Konstantin Shvachko made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #602 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/602/)
        MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache. Contributed by Priyo Mustafi.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #602 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/602/ ) MAPREDUCE-1915 . Fix IndexOutOfBoundsException in IndexCache. Contributed by Priyo Mustafi.
        Hide
        Rares Vernica added a comment -

        Priyo and Konstantin, thanks for taking care of this!

        Show
        Rares Vernica added a comment - Priyo and Konstantin, thanks for taking care of this!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        206d 22h 51m 1 Priyo Mustafi 28/Jan/11 21:57
        Open Open Patch Available Patch Available
        4m 30s 2 Priyo Mustafi 28/Jan/11 21:58
        Patch Available Patch Available Resolved Resolved
        7d 19m 1 Konstantin Shvachko 04/Feb/11 22:18

          People

          • Assignee:
            Priyo Mustafi
            Reporter:
            Rares Vernica
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development