Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1709

HDFS CompressedDataStream doesn't support serializer parameter

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.4.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      When configuring the HDFS sink you can specify a serializer parameter which is hooked into the append method of the sink.

      The HDFSCompressedDataStream does not respect this parameter and the specified serializer is not used.

      1. FLUME-1709.patch
        9 kB
        Cameron Gandevia

        Issue Links

          Activity

          Hide
          mpercy Mike Percy added a comment -

          Nice. I would like to get this in but am a bit swamped... wondering if anybody else has time to review this at the moment?

          Show
          mpercy Mike Percy added a comment - Nice. I would like to get this in but am a bit swamped... wondering if anybody else has time to review this at the moment?
          Hide
          brocknoland Brock Noland added a comment -

          I can look at this today.

          Show
          brocknoland Brock Noland added a comment - I can look at this today.
          Hide
          brocknoland Brock Noland added a comment -

          Excellent work Cameron! I have committed to trunk and 1.4. Thanks again!

          Show
          brocknoland Brock Noland added a comment - Excellent work Cameron! I have committed to trunk and 1.4. Thanks again!
          Hide
          hudson Hudson added a comment -

          Integrated in flume-trunk #335 (See https://builds.apache.org/job/flume-trunk/335/)
          FLUME-1709: HDFS CompressedDataStream doesn't support serializer parameter (Revision 59d0329a6ba0db46e012e41ebf695ac65fbf71f8)

          Result = SUCCESS
          brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=59d0329a6ba0db46e012e41ebf695ac65fbf71f8
          Files :

          • flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSCompressedDataStream.java
          • flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java
          Show
          hudson Hudson added a comment - Integrated in flume-trunk #335 (See https://builds.apache.org/job/flume-trunk/335/ ) FLUME-1709 : HDFS CompressedDataStream doesn't support serializer parameter (Revision 59d0329a6ba0db46e012e41ebf695ac65fbf71f8) Result = SUCCESS brock : http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=59d0329a6ba0db46e012e41ebf695ac65fbf71f8 Files : flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSCompressedDataStream.java flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java
          Hide
          mpercy Mike Percy added a comment -

          Cameron: thanks for the fix!
          Brock: thanks for the review!

          Show
          mpercy Mike Percy added a comment - Cameron: thanks for the fix! Brock: thanks for the review!

            People

            • Assignee:
              gnoremac Cameron Gandevia
              Reporter:
              gnoremac Cameron Gandevia
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development