Index: src/docbkx/performance.xml
===================================================================
--- src/docbkx/performance.xml (revision 1197305)
+++ src/docbkx/performance.xml (working copy)
@@ -353,6 +353,18 @@
rows at a time to the client to be processed. There is a cost/benefit to
have the cache value be large because it costs more in memory for both
client and RegionServer, so bigger isn't always better.
+
+ Scan Caching in MapReduce Jobs
+ Scan settings in MapReduce jobs deserve special attention. Timeouts can result (e.g., UnknownScannerException)
+ in Map tasks if it takes longer to process a batch of records before the client goes back to the RegionServer for the
+ next set of data. This problem can occur because there is non-trivial processing occuring per row. If you process
+ rows quickly, set caching higher. If you process rows more slowly (e.g., lots of transformations per row, writes),
+ then set caching lower.
+
+ Timeouts can also happen in a non-MapReduce use case (i.e., single threaded HBase client doing a Scan), but the
+ processing that is often performed in MapReduce jobs tends to exacerbate this issue.
+
+
Scan Attribute Selection
Index: src/docbkx/book.xml
===================================================================
--- src/docbkx/book.xml (revision 1197305)
+++ src/docbkx/book.xml (working copy)
@@ -567,7 +567,7 @@
Cardinality of ColumnFamilies
Where multiple ColumnFamilies exist in a single table, be aware of the cardinality (i.e., number of rows).
- If ColumnFamilyA has 1000,000 rows and ColumnFamilyB has 1 billion rows, ColumnFamilyA's data will likely be spread
+ If ColumnFamilyA has 1 million rows and ColumnFamilyB has 1 billion rows, ColumnFamilyA's data will likely be spread
across many, many regions (and RegionServers). This makes mass scans for ColumnFamilyA less efficient.
Index: src/docbkx/troubleshooting.xml
===================================================================
--- src/docbkx/troubleshooting.xml (revision 1197305)
+++ src/docbkx/troubleshooting.xml (working copy)
@@ -464,12 +464,14 @@
For more information on the HBase client, see .
- ScannerTimeoutException
+ ScannerTimeoutException or UnknownScannerException
This is thrown if the time between RPC calls from the client to RegionServer exceeds the scan timeout.
For example, if Scan.setCaching is set to 500, then there will be an RPC call to fetch the next batch of rows every 500 .next() calls on the ResultScanner
because data is being transferred in blocks of 500 rows to the client. Reducing the setCaching value may be an option, but setting this value too low makes for inefficient
processing on numbers of rows.
+ See .
+
Shell or client application throws lots of scary exceptions during normal operation