[HBASE-17849] PE tool random read is not totally random

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Performance, test
• Hadoop Flags: Reviewed

Release Note

When randomRead and randomSeekScan is used with PE tool, now we allow using both --size and --rows. The --size specifies the total size of the data (the range) on which the reads should be performed and --rows specifies the number of rows to be read by each client with in that range.

• Assignee: ramkrishna.s.vasudevan
• Reporter: ramkrishna.s.vasudevan
• Created: 29/Mar/17 18:11
• Updated: 06/Apr/18 04:57
• Resolved: 07/Jun/17 06:02

Description

Recently we were using the PE tool for doing some bucket cache related performance tests. One thing that we noted was that the way the random read works is not totally random.

Suppose we load 200G of data using --size param and then we use --rows=500000 to do the randomRead. The assumption was among the 200G of data it could generate randomly 500000 row keys to do the reads.
But it so happens that the PE tool generates random rows only on those set of row keys which falls under the first 500000 rows.
This was quite evident when we tried to use HBASE-15314 in our testing. Suppose we split the bucket cache of size 200G into 2 files each 100G the randomReads with --rows=500000 always lands in the first file and not in the 2nd file. Better to make PE purely random.

Issue Links

• duplicates: HBASE-13708 PE - Add option to specify the range