[HBASE-28482] Reverse scan with tags throws ArrayIndexOutOfBoundsException with DBE

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9
• Component/s: HFile
• Labels: pull-request-available
• Assignee: Vineet Kumar Maheshwari
• Reporter: Vineet Kumar Maheshwari
• Created: 03/Apr/24 04:20
• Updated: 29/Apr/24 21:08
• Resolved: 28/Apr/24 15:20

Description

Facing ArrayIndexOutOfBoundsException when performing reverse scan on a table with 30K+ records in single hfile.

Exception is happening  when block changes during seekBefore call.

Caused by: java.lang.ArrayIndexOutOfBoundsException
    at org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1326)
    at org.apache.hadoop.hbase.nio.SingleByteBuff.get(SingleByteBuff.java:213)
    at org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decode(DiffKeyDeltaEncoder.java:431)
    at org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder$DiffSeekerStateBufferedEncodedSeeker.decodeNext(DiffKeyDeltaEncoder.java:502)
    at org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.seekToKeyInBlock(BufferedDataBlockEncoder.java:1012)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$EncodedScanner.loadBlockAndSeekToKey(HFileReaderImpl.java:1605)
    at org.apache.hadoop.hbase.io.hfile.HFileReaderImpl$HFileScannerImpl.seekBefore(HFileReaderImpl.java:719)
    at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekBeforeAndSaveKeyToPreviousRow(StoreFileScanner.java:645)
    at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRowWithoutHint(StoreFileScanner.java:570)
    at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekToPreviousRow(StoreFileScanner.java:506)
    at org.apache.hadoop.hbase.regionserver.ReversedKeyValueHeap.next(ReversedKeyValueHeap.java:126)
    at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:693)
    at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:151)

 

Steps to reproduce:

Create a table with DataBlockEncoding.DIFF and block size as 1024, write some 30K+ puts with setTTL, then do a reverse scan.

@Test
public void testReverseScanWithDBEWhenCurrentBlockUpdates() throws IOException {
byte[] family = Bytes.toBytes("0");

Configuration conf = new Configuration(TEST_UTIL.getConfiguration());
conf.setInt(HConstants.HBASE_CLIENT_RETRIES_NUMBER, 1);

try (Connection connection = ConnectionFactory.createConnection(conf)) {
testReverseScanWithDBE(connection, DataBlockEncoding.DIFF, family, 1024, 30000);
for (DataBlockEncoding encoding : DataBlockEncoding.values()) {
testReverseScanWithDBE(connection, encoding, family, 1024, 30000);
}
}
}

private void testReverseScanWithDBE(Connection conn, DataBlockEncoding encoding, byte[] family, int blockSize, int maxRows)
throws IOException {
LOG.info("Running test with DBE={}", encoding);
TableName tableName = TableName.valueOf(TEST_NAME.getMethodName() + "-" + encoding);
TEST_UTIL.createTable(TableDescriptorBuilder.newBuilder(tableName)
.setColumnFamily(
ColumnFamilyDescriptorBuilder.newBuilder(family).setDataBlockEncoding(encoding).setBlocksize(blockSize).build())
.build(), null);

Table table = conn.getTable(tableName);

byte[] val1 = new byte[10];
byte[] val2 = new byte[10];
Bytes.random(val1);
Bytes.random(val2);

for (int i = 0; i < maxRows; i++) {
table.put(new Put(Bytes.toBytes(i)).addColumn(family, Bytes.toBytes(1), val1)
.addColumn(family, Bytes.toBytes(2), val2).setTTL(600_000));
}

TEST_UTIL.flush(table.getName());

Scan scan = new Scan();
scan.setReversed(true);

try (ResultScanner scanner = table.getScanner(scan)) {
for (int i = maxRows - 1; i >= 0; i--) {
Result row = scanner.next();
assertEquals(2, row.size());

Cell cell1 = row.getColumnLatestCell(family, Bytes.toBytes(1));
assertTrue(CellUtil.matchingRows(cell1, Bytes.toBytes(i)));
assertTrue(CellUtil.matchingValue(cell1, val1));

Cell cell2 = row.getColumnLatestCell(family, Bytes.toBytes(2));
assertTrue(CellUtil.matchingRows(cell2, Bytes.toBytes(i)));
assertTrue(CellUtil.matchingValue(cell2, val2));
}
}

}

 

HBASE-27580 has fixed the issue for seekBefore in rewind function flow, but similar change is required in setCurrentBlock function flow also.

Issue Links

• is related to: HBASE-27580 Reverse scan over rows with tags throw exceptions when using DataBlockEncoding
• links to: GitHub Pull Request #5792