[FLUME-2535] DEFAULT_BUF_SIZE hits special unicode character on the last byte truncates the event stream

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.4.0
• Component/s: Sinks+Sources
• Assignee: Unassigned
• Reporter: Asim Zafir
• Created: 04/Nov/14 10:59
• Updated: 04/Nov/14 11:45

Description

we have come to acknowledge a special situation and that is if the DEFAULT_BUF_SIZE mentioned on line 54 of org/apache/flume/serialization/ResettableFileInputStream.java happens to hit special unicode character (Latin Small Letter E with Circumflex
Unicode number: U+00EA
HTML-code: ê) anything includring and beyond this on the entire stream event will be truncated.

Issue Links

• is related to: FLUME-2360 Spooling Source: LineDeserializer incorrectly stops reading events if UTF-8 char occurs at end of buffer
• is related to: FLUME-2215 ResettableFileInputStream can't support ucs-4 character