Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1915

IndexCache - getIndexInformation - check reduce index Out Of Bounds

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When checking if the "reduce" index is out of bounds the check should be

      info.mapSpillRecord.size() <= reduce

      instead of:

      info.mapSpillRecord.size() < reduce

      Not a big bug since an Out Of Bounds is thrown downstream anyway.

      1. MAPREDUCE-1915.patch
        0.6 kB
        Rares Vernica
      2. MAPREDUCE-1915_1.patch
        3 kB
        Priyo Mustafi

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #33 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/33/ )
        Hide
        Rares Vernica added a comment -

        Priyo and Konstantin, thanks for taking care of this!

        Show
        Rares Vernica added a comment - Priyo and Konstantin, thanks for taking care of this!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #602 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/602/)
        MAPREDUCE-1915. Fix IndexOutOfBoundsException in IndexCache. Contributed by Priyo Mustafi.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #602 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/602/ ) MAPREDUCE-1915 . Fix IndexOutOfBoundsException in IndexCache. Contributed by Priyo Mustafi.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this. Thank you Priyo.

        Show
        Konstantin Shvachko added a comment - I just committed this. Thank you Priyo.
        Hide
        Priyo Mustafi added a comment -

        Got a +1 on release audit as well. It was an environmental issue. Patch-test and test all passed now.

        Show
        Priyo Mustafi added a comment - Got a +1 on release audit as well. It was an environmental issue. Patch-test and test all passed now.
        Hide
        Priyo Mustafi added a comment -

        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] -1 release audit. The applied patch generated 20 release audit warnings (more than the trunk's current 1 warnings).
        [exec]
        [exec] +1 system test framework. The patch passed system test framework compile.

        Show
        Priyo Mustafi added a comment - [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 release audit. The applied patch generated 20 release audit warnings (more than the trunk's current 1 warnings). [exec] [exec] +1 system test framework. The patch passed system test framework compile.
        Hide
        Priyo Mustafi added a comment -

        Added to 0.22

        Show
        Priyo Mustafi added a comment - Added to 0.22
        Hide
        Konstantin Shvachko added a comment -

        +1 I liked the test case. Please assign it to 0.22 as a bug fix.

        Show
        Konstantin Shvachko added a comment - +1 I liked the test case. Please assign it to 0.22 as a bug fix.
        Hide
        Priyo Mustafi added a comment -

        Added testcase. The latest attachment contains both the fix and the testcase.

        Show
        Priyo Mustafi added a comment - Added testcase. The latest attachment contains both the fix and the testcase.
        Hide
        Priyo Mustafi added a comment -

        Hi Rares,
        The fix makes sense. I am writing a testcase for it but then I am not sure when this scenario is going to ever occur. Looks like you noticed an OutOfBoundException but under what circumstance?

        Show
        Priyo Mustafi added a comment - Hi Rares, The fix makes sense. I am writing a testcase for it but then I am not sure when this scenario is going to ever occur. Looks like you noticed an OutOfBoundException but under what circumstance?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448730/MAPREDUCE-1915.patch
        against trunk revision 960534.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448730/MAPREDUCE-1915.patch against trunk revision 960534. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/286/console This message is automatically generated.

          People

          • Assignee:
            Priyo Mustafi
            Reporter:
            Rares Vernica
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development