Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1915

Enhance NettyAvroRpcClient and the use of NettyServer to optionally use compression

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.4.0
    • None
    • None

    Description

      This JIRA will update AvroSource and NettyAvroRpcClient to optionally use ZlibEncoder and ZlibDecoder to compress communications between themselves.

      Attachments

        1. FLUME-1915.patch
          26 kB
          Theodore michael Malaska
        2. FLUME-1915-1.patch
          40 kB
          Theodore michael Malaska

        Issue Links

          Activity

            Added compression options for both client and server.

            Note compress has be turn on both client and server for it to work.

            Client can have compression with out compression if compression level is set to 0

            ted.m Theodore michael Malaska added a comment - Added compression options for both client and server. Note compress has be turn on both client and server for it to work. Client can have compression with out compression if compression level is set to 0

            Updated based on review

            ted.m Theodore michael Malaska added a comment - Updated based on review
            mpercy Mike Percy added a comment -

            +1

            mpercy Mike Percy added a comment - +1
            mpercy Mike Percy added a comment -

            Pushed to trunk and flume-1.4 branches. Thanks for the patch, Ted!

            mpercy Mike Percy added a comment - Pushed to trunk and flume-1.4 branches. Thanks for the patch, Ted!
            hudson Hudson added a comment -

            Integrated in flume-trunk #369 (See https://builds.apache.org/job/flume-trunk/369/)
            FLUME-1915. Deflate compression support for the AvroSource, AvroSink, and Avro RPC Client. (Revision e72e559baa589787bcba233e29882e28e0ff43ef)

            Result = SUCCESS
            mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=e72e559baa589787bcba233e29882e28e0ff43ef
            Files :

            • flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java
            • flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java
            • flume-ng-doc/sphinx/FlumeUserGuide.rst
            • flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java
            • flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java
            • flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java
            • flume-ng-core/src/test/java/org/apache/flume/source/TestAvroSource.java
            • flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java
            • flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java
            hudson Hudson added a comment - Integrated in flume-trunk #369 (See https://builds.apache.org/job/flume-trunk/369/ ) FLUME-1915 . Deflate compression support for the AvroSource, AvroSink, and Avro RPC Client. (Revision e72e559baa589787bcba233e29882e28e0ff43ef) Result = SUCCESS mpercy : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=e72e559baa589787bcba233e29882e28e0ff43ef Files : flume-ng-core/src/main/java/org/apache/flume/sink/AbstractRpcSink.java flume-ng-core/src/main/java/org/apache/flume/source/AvroSource.java flume-ng-doc/sphinx/FlumeUserGuide.rst flume-ng-sdk/src/test/java/org/apache/flume/api/TestNettyAvroRpcClient.java flume-ng-sdk/src/test/java/org/apache/flume/api/RpcTestUtils.java flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java flume-ng-core/src/test/java/org/apache/flume/source/TestAvroSource.java flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java flume-ng-sdk/src/main/java/org/apache/flume/api/NettyAvroRpcClient.java

            People

              ted.m Theodore michael Malaska
              ted.m Theodore michael Malaska
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: