HBase
  1. HBase
  2. HBASE-2625

Make testDynamicBloom()'s "randomness" deterministic

    Details

    • Type: Test Test
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.90.0
    • Fix Version/s: 0.90.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Had a failure with testDynamicBloom on Hudson today. Will investigate, however it would be nice to reproduce the problem to make sure it's not the fault of my test assumptions. I plan to seed the Random number generator with the current time and print that out for post-mortem analysis.

      1. hbase-2625.patch
        2 kB
        Nicolas Spiegelberg

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2838 (See https://builds.apache.org/job/HBase-TRUNK/2838/)
        HBASE-2625 Avoid byte buffer allocations when reading a value from a Result object (Tudor Scurtu) (Revision 1333159)

        Result = SUCCESS
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java
        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestResult.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2838 (See https://builds.apache.org/job/HBase-TRUNK/2838/ ) HBASE-2625 Avoid byte buffer allocations when reading a value from a Result object (Tudor Scurtu) (Revision 1333159) Result = SUCCESS tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/KeyValue.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Result.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestKeyValue.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestResult.java
        Hide
        stack added a comment -

        @Nicolas Here is what I did: HBASE-2653 I removed the filter and the test that used it.

        Show
        stack added a comment - @Nicolas Here is what I did: HBASE-2653 I removed the filter and the test that used it.
        Hide
        Nicolas Spiegelberg added a comment -

        Did you cut dynamic blooms or all the bloom tests? I can't see TestByteBloomFilter.java in the trunk. You could change testDynamicBloom() to use a static bloom instead and rename to 'testRandomEntries' or something like that.

        Show
        Nicolas Spiegelberg added a comment - Did you cut dynamic blooms or all the bloom tests? I can't see TestByteBloomFilter.java in the trunk. You could change testDynamicBloom() to use a static bloom instead and rename to 'testRandomEntries' or something like that.
        Hide
        stack added a comment -

        OK Nicolas. I'm going to cut Dynamic blooms especially as they seem to fail up in hudson sometimes: http://hudson.zones.apache.org/hudson/job/HBase-TRUNK/1282/testReport/org.apache.hadoop.hbase.util/TestByteBloomFilter/testDynamicBloom/

        Show
        stack added a comment - OK Nicolas. I'm going to cut Dynamic blooms especially as they seem to fail up in hudson sometimes: http://hudson.zones.apache.org/hudson/job/HBase-TRUNK/1282/testReport/org.apache.hadoop.hbase.util/TestByteBloomFilter/testDynamicBloom/
        Hide
        Nicolas Spiegelberg added a comment -

        We don't use Dynamic blooms right now. We can take them out as dead code. The random (vs. mod 2) test might be good to keep in and switch from using dynamic blooms to static blooms.

        Show
        Nicolas Spiegelberg added a comment - We don't use Dynamic blooms right now. We can take them out as dead code. The random (vs. mod 2) test might be good to keep in and switch from using dynamic blooms to static blooms.
        Hide
        stack added a comment -

        Applied. Thanks Nicolas (Do we even use DynamicBloomFilters?)

        Show
        stack added a comment - Applied. Thanks Nicolas (Do we even use DynamicBloomFilters?)
        Hide
        Nicolas Spiegelberg added a comment -

        Note that I think I also addressed the testDynamicBloom() flakiness in Hudson as well and documented the test assumptions a little better. If the keyCount exceeds 2000 keys, 3 blooms will be created. This will increase the theoretical error rate to 6*keyInterval*err, which is higher than the tolerance. Only a couple lines in 1 file, so I didn't think it was big enough to slap in review board.

        Show
        Nicolas Spiegelberg added a comment - Note that I think I also addressed the testDynamicBloom() flakiness in Hudson as well and documented the test assumptions a little better. If the keyCount exceeds 2000 keys, 3 blooms will be created. This will increase the theoretical error rate to 6*keyInterval*err, which is higher than the tolerance. Only a couple lines in 1 file, so I didn't think it was big enough to slap in review board.

          People

          • Assignee:
            Nicolas Spiegelberg
            Reporter:
            Nicolas Spiegelberg
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development