Details
-
Improvement
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
1.5.0
-
None
-
None
Description
compressing events in batch is much more efficient than compressing one by one.
I set hdfs.batchSize to 200000, when I use appendBatch() in BucketWriter, the append operation cost less than 1 seconds, while one by one might cost 10 seconds.