[FLUME-486] When using taildir in a single node configuration records are being lost

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 0.9.1
• Fix Version/s: 0.9.5
• Component/s: Node, Sinks+Sources
• Labels: events flume missing records sink tailDir text

Environment

CentOS 5.5 Sun JDK 1.6.0_b22

• Assignee: Unassigned
• Reporter: Disabled imported user
• Created: 22/Jan/11 07:07
• Updated: 05/Nov/14 09:29
• Resolved: 05/Nov/14 09:29

Description

When using tailDir to ingest file using flumes we're seeing large numbers of records go missing from the beginning of the file. If we use the text source at the same file all records ingest successfully. The workflow we're using is very simple, one node is used per source and sink. The node reads the data using tailDir and writes out using escapedCustomDfs.