Details
-
Improvement
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
0.95.2
-
None
-
Reviewed
Description
If the KV is large or Hlog is closed with high-pressure putting, we found memstore is often above the high water mark and block the putting.
So should we enable multi-thread for Memstore Flush?
Some performance test data for reference,
1.test environment :
random writting;upper memstore limit 5.6GB;lower memstore limit 4.8GB;400 regions per regionserver;row len=50 bytes, value len=1024 bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler per client for writing
2.test results:
one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per regionserver, appears many aboveGlobalMemstoreLimit blocking
two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per regionserver,
200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per regionserver, Flush:18.6MB/s per regionserver
Attachments
Attachments
Issue Links
- relates to
-
HBASE-8752 Backport HBASE-6466 to 0.94
- Closed