[FLUME-2545] How to convert *.bz2.tmp to *.bz2 file after restating the instance

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.4.0
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Arun Gujjar
• Created: 11/Nov/14 19:54
• Updated: 29/Mar/17 01:14

Description

Hi,

Whenever we restart flume agent it creates a new HDFS file and start writing the data into that file. The earlier file which was created will still be left as *bz2.tmp and from HIVE queries we found that we were not able to read the data from this file.

Here are the two questions I have .
1. Could you please suggest how we can convert this bz2.tmp to bz2 file? because we loose this data i.e. present in bz2.tmp file today.
2. Is there as way to configure flume to start writing the data into the existing bz2.tmp file instead of creating a new file?
3.