Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Abandoned
-
None
-
None
-
None
-
None
Description
The new write ahead log mechanism seems to batch too few mutations when running inside the disruptor. As we scaled our load up (many threads with small writes), we saw the number of hdfs sync operations grow in concert with the number of writes. Generally, one would expect the size of the batches to grow but the number of actual sync operations to settle.
