Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5402

PerformanceEvaluation creates the wrong number of rows in randomWrite

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: test
    • Labels:

      Description

      The command line 'hbase org.apache.hadoop.hbase.PerformanceEvaluation randomWrite 10' should result in a table with 10 * (1024 * 1024) rows (so 10485760). Instead what happens is that the randomWrite job reports writing that many rows (exactly) but running rowcounter against the table reveals only e.g 6549899 rows. A second attempt to build the table produced slightly different results (e.g. 6627689). I see a similar discrepancy when using 50 instead of 10 clients (~35% smaller than expected).

      Further experimentation reveals that the problem is key collision - by removing the % totalRows in getRandomRow I saw a reduction in collisions (table was ~8M rows instead of 6.6M). Replacing the random row key with UUIDs instead of Integers solved the problem and produced exactly 10485760 rows. But that makes the key size 16 bytes instead of the current 10, so I'm not sure that's an acceptable solution.

      Here's the UUID code I used:

      public static byte[] format(final UUID uuid) {
      long msb = uuid.getMostSignificantBits();
      long lsb = uuid.getLeastSignificantBits();
      byte[] buffer = new byte[16];

      for (int i = 0; i < 8; i++)

      { buffer[i] = (byte) (msb >>> 8 * (7 - i)); }

      for (int i = 8; i < 16; i++)

      { buffer[i] = (byte) (lsb >>> 8 * (7 - i)); }

      return buffer;
      }

      which is invoked within getRandomRow with

      return format(UUID.randomUUID());

        Activity

        Hide
        clehene Cosmin Lehene added a comment -

        Closing it with resolution later. It may make sense to a have a deterministic, invertible hash, in case someone wants to play with it.

        Show
        clehene Cosmin Lehene added a comment - Closing it with resolution later. It may make sense to a have a deterministic, invertible hash, in case someone wants to play with it.
        Hide
        oliver_meyn Oliver Meyn added a comment -

        As nobody else has commented on this issue I don't think it's worth debating further - I'm fine with Close as Invalid.

        Show
        oliver_meyn Oliver Meyn added a comment - As nobody else has commented on this issue I don't think it's worth debating further - I'm fine with Close as Invalid.
        Hide
        clehene Cosmin Lehene added a comment -

        [~oliver_meyn`], Andrew Purtell this seems a nice to have. Here's the current code that generates the key

          static byte [] getRandomRow(final Random random, final int totalRows) {
            return format(random.nextInt(Integer.MAX_VALUE) % totalRows);
          }
        
        Show
        clehene Cosmin Lehene added a comment - [~oliver_meyn`] , Andrew Purtell this seems a nice to have. Here's the current code that generates the key static byte [] getRandomRow( final Random random, final int totalRows) { return format(random.nextInt( Integer .MAX_VALUE) % totalRows); }
        Hide
        oliver@mineallmeyn.com Oliver Meyn added a comment -

        JD has succinctly dismantled my UUID argument, so those are out And I hadn't thought through the caching issue, so obviously need to learn some more. I like Todd's idea of each mapper randomizing some fixed range of keys - that buys a predictable number of rows and also a defined key space for doing random reads.

        Show
        oliver@mineallmeyn.com Oliver Meyn added a comment - JD has succinctly dismantled my UUID argument, so those are out And I hadn't thought through the caching issue, so obviously need to learn some more. I like Todd's idea of each mapper randomizing some fixed range of keys - that buys a predictable number of rows and also a defined key space for doing random reads.
        Hide
        tlipcon Todd Lipcon added a comment -

        Why don't we change it so that, instead of random, it just counts from 1 to 1M but puts the bits through some kind of blender? Easiest is to reverse the bit order, but could also do more creative swapping. Each mapper could also take its ascribed range of keys, break it into 100 subranges, and randomize each of the subranges.

        Show
        tlipcon Todd Lipcon added a comment - Why don't we change it so that, instead of random, it just counts from 1 to 1M but puts the bits through some kind of blender? Easiest is to reverse the bit order, but could also do more creative swapping. Each mapper could also take its ascribed range of keys, break it into 100 subranges, and randomize each of the subranges.
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        The problem is that the resulting table is what is used in any subsequent scan tests using PE, which are then double reading some rows, rather than reading every row once

        Following this logic, how would a random read test work with keys that are UUIDs? You'll have to be lucky to get a couple of hits

        This is counter intuitive, and also introduces the possibility of cache hits, which I think is not what is expected by users doing a scan test.

        Considering that blocks are 64KB and rows are ~1.5KB (keys+value), cache hits is going to happen no matter what.

        Show
        jdcryans Jean-Daniel Cryans added a comment - The problem is that the resulting table is what is used in any subsequent scan tests using PE, which are then double reading some rows, rather than reading every row once Following this logic, how would a random read test work with keys that are UUIDs? You'll have to be lucky to get a couple of hits This is counter intuitive, and also introduces the possibility of cache hits, which I think is not what is expected by users doing a scan test. Considering that blocks are 64KB and rows are ~1.5KB (keys+value), cache hits is going to happen no matter what.
        Hide
        oliver@mineallmeyn.com Oliver Meyn added a comment -

        I agree that the test itself is accurate in as much as it writes the number of rows it says it will (given multiple versions). The problem is that the resulting table is what is used in any subsequent scan tests using PE, which are then double reading some rows, rather than reading every row once. This is counter intuitive, and also introduces the possibility of cache hits, which I think is not what is expected by users doing a scan test.

        Show
        oliver@mineallmeyn.com Oliver Meyn added a comment - I agree that the test itself is accurate in as much as it writes the number of rows it says it will (given multiple versions). The problem is that the resulting table is what is used in any subsequent scan tests using PE, which are then double reading some rows, rather than reading every row once. This is counter intuitive, and also introduces the possibility of cache hits, which I think is not what is expected by users doing a scan test.
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        I don't see this is an issue, it does create the right amount of rows if you consider counting versions.

        Show
        jdcryans Jean-Daniel Cryans added a comment - I don't see this is an issue, it does create the right amount of rows if you consider counting versions.

          People

          • Assignee:
            Unassigned
            Reporter:
            oliver@mineallmeyn.com Oliver Meyn
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development