[HBASE-13362] Set max result size from client only (like scanner caching).

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.1, 1.1.0, 0.98.12, 2.0.0
• Hadoop Flags: Reviewed

Release Note

This introduces a new config option: hbase.server.scanner.max.result.size
This setting enforces a maximum result size (in bytes), when reached the server will return the results is has so far.
This is a safety setting and should be kept large. The default is inifinite in 0.98 and 1.0.x and 100mb in 1.1 and later.

Use hbase.client.scanner.max.result.size instead to enforce practical chunk sizes of a few mb (defaults to 2mb)

• Assignee: Lars Hofhansl
• Reporter: Lars Hofhansl
• Created: 29/Mar/15 21:06
• Updated: 28/Apr/15 19:18
• Resolved: 09/Apr/15 05:03

Description

With the recent problems we've been seeing client/server result size mismatch, I was thinking: Why was this not a problem with scanner caching?

There are two reasons:

1. number of rows is easy to calculate (and we did it correctly)
2. caching is only controlled from the client, never set on the server alone

We did fix both #1 and #2 in HBASE-13262.

Still, I'd like to discuss the following:

• default the client sent max result size to 2mb
• remove any server only result sizing
• continue to use hbase.client.scanner.max.result.size but enforce it via the client only (as the name implies anyway).

Comments? Concerns?

Issue Links

• is related to: HBASE-13527 The default value for hbase.client.scanner.max.result.size is never actually set on Scans