Flume
  1. Flume
  2. FLUME-1915

Enhance NettyAvroRpcClient and the use of NettyServer to optionally use compression

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.4.0
    • Component/s: None
    • Labels:
      None

      Description

      This JIRA will update AvroSource and NettyAvroRpcClient to optionally use ZlibEncoder and ZlibDecoder to compress communications between themselves.

      1. FLUME-1915-1.patch
        40 kB
        Ted Malaska
      2. FLUME-1915.patch
        26 kB
        Ted Malaska

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in flume-trunk #369 (See https://builds.apache.org/job/flume-trunk/369/)
          FLUME-1915. Deflate compression support for the AvroSource, AvroSink, and Avro RPC Client. (Revision e72e559baa589787bcba233e29882e28e0ff43ef)

          Result = SUCCESS
          mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=e72e559baa589787bcba233e29882e28e0ff43ef
          Files :

          • flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java
          • flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
          • flume-ng-doc/sphinx/FlumeUserGuide.rst
          • flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
          • flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java
          • flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java
          • flume-ng-core/src/test/java/org/apache/flume/source/TestAvroSource.java
          • flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
          • flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
          Show
          Hudson added a comment - Integrated in flume-trunk #369 (See https://builds.apache.org/job/flume-trunk/369/ ) FLUME-1915 . Deflate compression support for the AvroSource, AvroSink, and Avro RPC Client. (Revision e72e559baa589787bcba233e29882e28e0ff43ef) Result = SUCCESS mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=e72e559baa589787bcba233e29882e28e0ff43ef Files : flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java flume-ng-doc/sphinx/FlumeUserGuide.rst flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java flume-ng-core/src/test/java/org/apache/flume/source/TestAvroSource.java flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
          Hide
          Mike Percy added a comment -

          Pushed to trunk and flume-1.4 branches. Thanks for the patch, Ted!

          Show
          Mike Percy added a comment - Pushed to trunk and flume-1.4 branches. Thanks for the patch, Ted!
          Hide
          Mike Percy added a comment -

          +1

          Show
          Mike Percy added a comment - +1
          Hide
          Ted Malaska added a comment -

          Updated based on review

          Show
          Ted Malaska added a comment - Updated based on review
          Hide
          Ted Malaska added a comment -

          Added compression options for both client and server.

          Note compress has be turn on both client and server for it to work.

          Client can have compression with out compression if compression level is set to 0

          Show
          Ted Malaska added a comment - Added compression options for both client and server. Note compress has be turn on both client and server for it to work. Client can have compression with out compression if compression level is set to 0

            People

            • Assignee:
              Ted Malaska
              Reporter:
              Ted Malaska
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development