HBase
  1. HBase
  2. HBASE-10552

HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.96.2, 0.98.1, 0.99.0, 0.94.17
    • Component/s: None
    • Labels:
      None

      Description

      GaussianRandomReadBenchmark generates seek keys by using a Gaussian distribution with the mean of N/2 and a sigma of N/10 (N = number of rows used) and using this key directly to seek into the HFile. The HFile was seeded with keys from 0-N.

      This will fail if we ever generate a key < 0, which is rare, but by no means impossible. We need to clamp the min and max values to 0 and N, resp.

      1. 10552-0.94.txt
        1 kB
        Lars Hofhansl
      2. 10552-trunk.txt
        1 kB
        Lars Hofhansl

        Activity

        Hide
        Lars Hofhansl added a comment -

        Simple patch (for 0.94)

        Show
        Lars Hofhansl added a comment - Simple patch (for 0.94)
        Hide
        Ted Yu added a comment -

        +1

        Show
        Ted Yu added a comment - +1
        Hide
        Jean-Marc Spaggiari added a comment -

        +1 too. Does it need to be done in > 0.94 too?

        Show
        Jean-Marc Spaggiari added a comment - +1 too. Does it need to be done in > 0.94 too?
        Hide
        Lars Hofhansl added a comment -

        Trunk has a different fix for this:

              if (scanner.seekTo(gaussianRandomRowBytes) < 0) {
                LOG.info("Not able to seekTo " + new String(gaussianRandomRowBytes));
                return;
              }
        

        Which is a hack, IMHO. The reason we get -1 is because we generated a seek key before the first key of the file.

        Show
        Lars Hofhansl added a comment - Trunk has a different fix for this: if (scanner.seekTo(gaussianRandomRowBytes) < 0) { LOG.info( "Not able to seekTo " + new String (gaussianRandomRowBytes)); return ; } Which is a hack, IMHO. The reason we get -1 is because we generated a seek key before the first key of the file.
        Hide
        Lars Hofhansl added a comment -

        Proposed patch for trunk.
        (inlined max/min stuff; and remove the < 0 check, I'd rather see an exception)

        Show
        Lars Hofhansl added a comment - Proposed patch for trunk. (inlined max/min stuff; and remove the < 0 check, I'd rather see an exception)
        Hide
        Lars Hofhansl added a comment -

        Any objections?

        Show
        Lars Hofhansl added a comment - Any objections?
        Hide
        Jean-Marc Spaggiari added a comment -

        No objection on my side. +1 with trunk version too.

        Show
        Jean-Marc Spaggiari added a comment - No objection on my side. +1 with trunk version too.
        Hide
        Lars Hofhansl added a comment -

        Going to commit to all branches in a few.

        Show
        Lars Hofhansl added a comment - Going to commit to all branches in a few.
        Hide
        Lars Hofhansl added a comment -

        Committed to all branches.

        Show
        Lars Hofhansl added a comment - Committed to all branches.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94-security #415 (See https://builds.apache.org/job/HBase-0.94-security/415/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94-security #415 (See https://builds.apache.org/job/HBase-0.94-security/415/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #26 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/26/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-on-Hadoop-2 #26 (See https://builds.apache.org/job/HBase-0.94-on-Hadoop-2/26/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.94-JDK7 #55 (See https://builds.apache.org/job/HBase-0.94-JDK7/55/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.94-JDK7 #55 (See https://builds.apache.org/job/HBase-0.94-JDK7/55/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in hbase-0.96 #297 (See https://builds.apache.org/job/hbase-0.96/297/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569128)

        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in hbase-0.96 #297 (See https://builds.apache.org/job/hbase-0.96/297/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569128) /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #152 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/152/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569127)

        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #152 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/152/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569127) /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96-hadoop2 #206 (See https://builds.apache.org/job/hbase-0.96-hadoop2/206/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569128)

        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96-hadoop2 #206 (See https://builds.apache.org/job/hbase-0.96-hadoop2/206/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569128) /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-0.98 #163 (See https://builds.apache.org/job/HBase-0.98/163/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569127)

        • /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-0.98 #163 (See https://builds.apache.org/job/HBase-0.98/163/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569127) /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        ABORTED: Integrated in HBase-0.94 #1292 (See https://builds.apache.org/job/HBase-0.94/1292/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - ABORTED: Integrated in HBase-0.94 #1292 (See https://builds.apache.org/job/HBase-0.94/1292/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569129) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #4925 (See https://builds.apache.org/job/HBase-TRUNK/4925/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569126)

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #4925 (See https://builds.apache.org/job/HBase-TRUNK/4925/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569126) /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #93 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/93/)
        HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569126)

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #93 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/93/ ) HBASE-10552 HFilePerformanceEvaluation.GaussianRandomReadBenchmark fails sometimes. (larsh: rev 1569126) /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java

          People

          • Assignee:
            Unassigned
            Reporter:
            Lars Hofhansl
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development