[FLUME-2540] FLUME failed to rotate HDFS sink file from *.tmp to normal when timezone changed from PDT - PST on Nov 02, 2014

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4.0
• Component/s: Sinks+Sources

Environment

apache-flume-1.4.0-cdh4.5.0, zookeeper-3.4.5-cdh4.5.0

• Assignee: Unassigned
• Reporter: Arun Gujjar
• Created: 05/Nov/14 20:34
• Updated: 11/Nov/14 19:46

Description

I have 2 instance of FLUME installed in a given host which has a timezone set in GMT. The day when time changed from PDT - PST on Nov 02, 2014. One of the flume instance failed to convert sink FlumeData.xxxxxxxxx.bz2.tmp to a FlumeData.xxxxxxxxx.bz2 file, but the other instance had no issues of this timezone change.

The difference between these two instances is, the instance which failed to convert sink filename writes the data ~6mb per hour, but other instance writes ~100 kb per hour.

Could you please suggest what could be the reason and how to convert file from FlumeData.xxxxxxxxx.bz2.tmp to FlumeData.xxxxxxxxx.bz2 file