Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: v1.5.0
-
Component/s: None
-
Labels:None
Issue Links
- links to
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
+1
Show
Mike Percy
added a comment - +1
Pushed to trunk and flume-1.5 branches. Thanks for the patch Hari!
Show
Mike Percy
added a comment - Pushed to trunk and flume-1.5 branches. Thanks for the patch Hari!
FAILURE: Integrated in flume-trunk #507 (See https://builds.apache.org/job/flume-trunk/507/)
FLUME-2202. AsyncHBaseSink should coalesce increments to reduce RPC roundtrips (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c4e2129fd12f97303a1b8120a2ecf7da456e1b77)
- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/IncrementAsyncHBaseSerializer.java
- flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
- flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSinkConfigurationConstants.java
- flume-ng-doc/sphinx/FlumeUserGuide.rst
- flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestAsyncHBaseSink.java
Show
Hudson
added a comment - FAILURE: Integrated in flume-trunk #507 (See https://builds.apache.org/job/flume-trunk/507/ )
FLUME-2202 . AsyncHBaseSink should coalesce increments to reduce RPC roundtrips (mpercy: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=c4e2129fd12f97303a1b8120a2ecf7da456e1b77 )
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/IncrementAsyncHBaseSerializer.java
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/HBaseSinkConfigurationConstants.java
flume-ng-doc/sphinx/FlumeUserGuide.rst
flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestAsyncHBaseSink.java
Added a new config to enable coalescing increments per batch. Even though asynchbase has an API that can coalesce, that does not allow us to check how many increments are in the buffer, which we need to calculate how many callbacks to expect - which is used to calculate timeouts.