[FLUME-1096] Add support to pass headers through AvroCLIClient

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0
• Labels: avroclient
• Assignee: Inder SIngh
• Reporter: Inder SIngh
• Created: 03/Apr/12 11:52
• Updated: 24/Apr/12 13:16
• Resolved: 24/Apr/12 12:30

Description

Configured HDFSEvent Sink to generate file path's using regex based date format.

Config for hdfs-sink -> collector1.sinks.hdfs-sink1.hdfs.filePrefix = flume/%D
Hit NumberFormatException in BucketPath.replaceShorthand() as it reads timestamp from headers

Date date = new Date(Long.valueOf(headers.get("timestamp")));

java.lang.NumberFormatException: null
at java.lang.Long.parseLong(Long.java:375)
at java.lang.Long.valueOf(Long.java:525)
at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:185)
at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:219)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:341)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:65)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:148)
at java.lang.Thread.run(Thread.java:680)

Sending the events using - ./flume-ng avro-client --conf ../conf/ -H localhost -p 41415 -F ~/.bash_profile

Issue - AvroCLiClient isn't adding timestamp in headers