Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-635

Allow the compression codec to be specified in SequenceFiles

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.9.4
    • None
    • None

    Description

      The dfs sink writes sequence files but doesn't honour flume.collector.dfs.compress.codec. On the other hand, the customdfs sink supports compression (via flume.collector.dfs.compress.codec), but doesn't write SequenceFiles.

      This can be fixed by either changing the dfs sink to honour flume.collector.dfs.compress.codec (compression would be enabled by setting io.seqfile.compression.type), or by creating a SequenceFileOutputFormat for use with the customdfs sink.

      Attachments

        1. ASF.LICENSE.NOT.GRANTED--FLUME-635-1.patch
          7 kB
          Disabled imported user
        2. ASF.LICENSE.NOT.GRANTED--FLUME-635-2.patch
          14 kB
          Disabled imported user
        3. ASF.LICENSE.NOT.GRANTED--FLUME-635-2.patch
          14 kB
          Disabled imported user

        Issue Links

          Activity

            People

              flume_tom Disabled imported user
              flume_tom Disabled imported user
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: