[FLUME-881] Would be nice if HDFS Sink would automatically choose best writableFormat based on fileType

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0, 1.1.0
• Fix Version/s: 1.2.0
• Component/s: Sinks+Sources
• Assignee: Jarek Jarcec Cecho
• Reporter: Jarek Jarcec Cecho
• Created: 10/Dec/11 18:46
• Updated: 26/Mar/12 04:15
• Resolved: 26/Mar/12 03:32

Description

Currently user can specify both file type (SequenceFile, DataStream, ...) and writable format (Text ~ append new line character, Writable ~ serialize as is) of HDFS sink output file. Both properties have default values, however when user redefine only file type to DataStream, flume will still use default Writable writable format that will not append separate serialized events with new line character.

Would be nice if flume would choose default value of writable format based on current file type, so that for example for DataStream file type, default writable format would be Text instead of Writable. I believe that would lead to less confused users.

Issue Links

• is related to: FLUME-860 Combination of ExecSource and HDFS DataStream is removing end line characters