Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3139

SlivePartitioner generates negative partitions

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      SlivePartitioner.getPartition() returns negative partition numbers on some occasions, which is illegal.

      1. MR-3139-0.patch
        0.8 kB
        Jakob Homan

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #886 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/886/)
          MAPREDUCE-3139. SlivePartitioner generates negative partitions.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #886 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/886/ ) MAPREDUCE-3139 . SlivePartitioner generates negative partitions. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Build #79 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/79/)
          MAPREDUCE-3139. Merge from trunk to 0.23.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #79 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/79/ ) MAPREDUCE-3139 . Merge from trunk to 0.23. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #852 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/852/)
          MAPREDUCE-3139. SlivePartitioner generates negative partitions.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #852 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/852/ ) MAPREDUCE-3139 . SlivePartitioner generates negative partitions. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #65 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/65/)
          MAPREDUCE-3139. Merge from trunk to 0.23.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #65 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/65/ ) MAPREDUCE-3139 . Merge from trunk to 0.23. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-22-branch #87 (See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/87/)
          MAPREDUCE-3139. Merge from trunk to 0.22.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196782
          Files :

          • /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt
          • /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #87 (See https://builds.apache.org/job/Hadoop-Mapreduce-22-branch/87/ ) MAPREDUCE-3139 . Merge from trunk to 0.22. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196782 Files : /hadoop/common/branches/branch-0.22/mapreduce/CHANGES.txt /hadoop/common/branches/branch-0.22/mapreduce/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-0.23-Commit #137 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/137/)
          MAPREDUCE-3139. Merge from trunk to 0.23.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #137 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/137/ ) MAPREDUCE-3139 . Merge from trunk to 0.23. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Commit #138 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/138/)
          MAPREDUCE-3139. Merge from trunk to 0.23.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #138 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/138/ ) MAPREDUCE-3139 . Merge from trunk to 0.23. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1236 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1236/)
          MAPREDUCE-3139. SlivePartitioner generates negative partitions.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1236 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1236/ ) MAPREDUCE-3139 . SlivePartitioner generates negative partitions. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1311 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1311/)
          MAPREDUCE-3139. SlivePartitioner generates negative partitions.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1311 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1311/ ) MAPREDUCE-3139 . SlivePartitioner generates negative partitions. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-0.23-Commit #148 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/148/)
          MAPREDUCE-3139. Merge from trunk to 0.23.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #148 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/148/ ) MAPREDUCE-3139 . Merge from trunk to 0.23. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196783 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1258 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1258/)
          MAPREDUCE-3139. SlivePartitioner generates negative partitions.

          jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
          • /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1258 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1258/ ) MAPREDUCE-3139 . SlivePartitioner generates negative partitions. jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1196776 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/fs/slive/SlivePartitioner.java
          Hide
          Jakob Homan added a comment -

          I've committed this to 22-trunk. Resolving.

          Show
          Jakob Homan added a comment - I've committed this to 22-trunk. Resolving.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12501896/MR-3139-0.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1237//testReport/
          Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1237//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12501896/MR-3139-0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1237//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1237//console This message is automatically generated.
          Hide
          Konstantin Shvachko added a comment -

          +1 This solves the problem.

          Show
          Konstantin Shvachko added a comment - +1 This solves the problem.
          Hide
          Jakob Homan added a comment -

          Quick patch to guarantee hash value is always positive. No tests as not easily testable, but easily verifiable with math. After review will commit to 22, 23 and trunk.

          Show
          Jakob Homan added a comment - Quick patch to guarantee hash value is always positive. No tests as not easily testable, but easily verifiable with math. After review will commit to 22, 23 and trunk.
          Hide
          Konstantin Shvachko added a comment -

          See HDFS-2398.

          Show
          Konstantin Shvachko added a comment - See HDFS-2398 .

            People

            • Assignee:
              Jakob Homan
              Reporter:
              Konstantin Shvachko
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development