Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13202

Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: util
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Current implementation:
      return (vectorSize + 7) / 8;
      when vectorSize is 2147483647´╝łthe max value of Int), error :"java.lang.NegativeArraySizeException" will report
      the implementation might be changed
      return (int)(((long)vectorSize + 7) / 8);

      1. HADOOP-13202.01.patch
        2 kB
        Kai Sasaki
      2. HADOOP-13202.02.patch
        2 kB
        Kai Sasaki
      3. HADOOP-13202.03.patch
        2 kB
        Kai Sasaki

        Activity

        Hide
        lewuathe Kai Sasaki added a comment -

        Akira Ajisaka Thank you so much!

        Show
        lewuathe Kai Sasaki added a comment - Akira Ajisaka Thank you so much!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10115 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10115/)
        HADOOP-13202. Avoid possible overflow in (aajisaka: rev c2bcffb34ebe4399b523a44f69d23761648bd0c4)

        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/bloom/TestBloomFilters.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10115 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10115/ ) HADOOP-13202 . Avoid possible overflow in (aajisaka: rev c2bcffb34ebe4399b523a44f69d23761648bd0c4) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/bloom/TestBloomFilters.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to trunk, branch-2, branch-2.8, and branch-2.7. Thanks Kai Sasaki for the contribution and thanks zhengbing li for the report.

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to trunk, branch-2, branch-2.8, and branch-2.7. Thanks Kai Sasaki for the contribution and thanks zhengbing li for the report.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1, thanks Kai Sasaki.

        Show
        ajisakaa Akira Ajisaka added a comment - +1, thanks Kai Sasaki .
        Hide
        lewuathe Kai Sasaki added a comment -

        Akira Ajisaka Thanks for reviewing! I updated and it worked as expected, since test failure does not look like relate to this patch.

        Show
        lewuathe Kai Sasaki added a comment - Akira Ajisaka Thanks for reviewing! I updated and it worked as expected, since test failure does not look like relate to this patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 50s trunk passed
        +1 compile 6m 45s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 18s trunk passed
        +1 javadoc 0m 45s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 6m 42s the patch passed
        +1 javac 6m 42s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 51s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 27s the patch passed
        +1 javadoc 0m 45s the patch passed
        -1 unit 6m 50s hadoop-common in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        37m 4s



        Reason Tests
        Failed junit tests hadoop.ipc.TestIPC



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818482/HADOOP-13202.03.patch
        JIRA Issue HADOOP-13202
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ea7ae1ea4e7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 5b4a708
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 50s trunk passed +1 compile 6m 45s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 18s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 6m 42s the patch passed +1 javac 6m 42s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 27s the patch passed +1 javadoc 0m 45s the patch passed -1 unit 6m 50s hadoop-common in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 37m 4s Reason Tests Failed junit tests hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12818482/HADOOP-13202.03.patch JIRA Issue HADOOP-13202 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ea7ae1ea4e7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5b4a708 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10018/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Kai Sasaki for providing the patch.
        Would you replace Math.toIntExact with (int) for backporting to branch-2? branch-2 still supports JDK7, and I'm thinking (int) is fine because (long)(any int value) + 7 / 8 is always in [Integer.MIN_VALUE, Integer.MAX_VALUE].

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Kai Sasaki for providing the patch. Would you replace Math.toIntExact with (int) for backporting to branch-2? branch-2 still supports JDK7, and I'm thinking (int) is fine because (long)(any int value) + 7 / 8 is always in [Integer.MIN_VALUE, Integer.MAX_VALUE] .
        Hide
        lewuathe Kai Sasaki added a comment -

        Akira Ajisaka Sorry for bothering you but could you review this when you are available?
        Thanks.

        Show
        lewuathe Kai Sasaki added a comment - Akira Ajisaka Sorry for bothering you but could you review this when you are available? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 48s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 54s trunk passed
        +1 compile 7m 5s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 56s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 1m 29s trunk passed
        +1 javadoc 0m 53s trunk passed
        +1 mvninstall 0m 37s the patch passed
        +1 compile 7m 14s the patch passed
        +1 javac 7m 14s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 1m 3s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 31s the patch passed
        +1 javadoc 0m 54s the patch passed
        -1 unit 8m 14s hadoop-common in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        50m 48s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806648/HADOOP-13202.02.patch
        JIRA Issue HADOOP-13202
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 798bde9a32e3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e4022de
        Default Java 1.8.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 48s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 54s trunk passed +1 compile 7m 5s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 1m 29s trunk passed +1 javadoc 0m 53s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 7m 14s the patch passed +1 javac 7m 14s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 31s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 8m 14s hadoop-common in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 50m 48s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806648/HADOOP-13202.02.patch JIRA Issue HADOOP-13202 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 798bde9a32e3 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e4022de Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9605/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 39s trunk passed
        +1 compile 6m 55s trunk passed
        +1 checkstyle 0m 23s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 20s trunk passed
        +1 javadoc 0m 54s trunk passed
        +1 mvninstall 0m 48s the patch passed
        +1 compile 6m 41s the patch passed
        +1 javac 6m 41s the patch passed
        -1 checkstyle 0m 23s hadoop-common-project/hadoop-common: The patch generated 1 new + 47 unchanged - 0 fixed = 48 total (was 47)
        +1 mvnsite 0m 54s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 37s the patch passed
        +1 javadoc 0m 55s the patch passed
        -1 unit 7m 51s hadoop-common in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        37m 56s



        Reason Tests
        Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806365/HADOOP-13202.01.patch
        JIRA Issue HADOOP-13202
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9f96fb531c7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 77202fa
        Default Java 1.8.0_91
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 39s trunk passed +1 compile 6m 55s trunk passed +1 checkstyle 0m 23s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 20s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 6m 41s the patch passed +1 javac 6m 41s the patch passed -1 checkstyle 0m 23s hadoop-common-project/hadoop-common: The patch generated 1 new + 47 unchanged - 0 fixed = 48 total (was 47) +1 mvnsite 0m 54s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 37s the patch passed +1 javadoc 0m 55s the patch passed -1 unit 7m 51s hadoop-common in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 37m 56s Reason Tests Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12806365/HADOOP-13202.01.patch JIRA Issue HADOOP-13202 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9f96fb531c7e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 77202fa Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9587/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        lewuathe Kai Sasaki added a comment -

        Though Math.toIntExact is a JDK8 API, HADOOP-11858 set minimum version JDK8. So it seems okay from Hadoop 3 to use.

        Show
        lewuathe Kai Sasaki added a comment - Though Math.toIntExact is a JDK8 API, HADOOP-11858 set minimum version JDK8. So it seems okay from Hadoop 3 to use.

          People

          • Assignee:
            lewuathe Kai Sasaki
            Reporter:
            bing zhengbing li
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1h
              1h
              Remaining:
              Remaining Estimate - 1h
              1h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development