[FLUME-635] Allow the compression codec to be specified in SequenceFiles - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.9.4
Component/s: None
Labels:
None

Description

The dfs sink writes sequence files but doesn't honour flume.collector.dfs.compress.codec. On the other hand, the customdfs sink supports compression (via flume.collector.dfs.compress.codec), but doesn't write SequenceFiles.

This can be fixed by either changing the dfs sink to honour flume.collector.dfs.compress.codec (compression would be enabled by setting io.seqfile.compression.type), or by creating a SequenceFileOutputFormat for use with the customdfs sink.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

ASF.LICENSE.NOT.GRANTED--FLUME-635-1.patch
21/May/11 00:41
7 kB
Disabled imported user
ASF.LICENSE.NOT.GRANTED--FLUME-635-2.patch
31/May/11 00:35
14 kB
Disabled imported user
ASF.LICENSE.NOT.GRANTED--FLUME-635-2.patch
21/May/11 00:41
14 kB
Disabled imported user

Issue Links

blocks

FLUME-632 Support arguments in output format configuration

Closed

is duplicated by

FLUME-506 Sink that writes output to a sequenceFile in hdfs

Resolved

is related to

FLUME-641 Support aliases for compression codecs

Closed

Activity

People

Assignee:: Disabled imported user

Reporter:: Disabled imported user

Votes:: 0 Vote for this issue

Watchers:: 0 Start watching this issue

Dates

Created:: 17/May/11 21:57

Updated:: 19/Aug/11 03:04

Resolved:: 31/May/11 21:08