[IMPALA-271] Sequence file spews bad status "Incomplete read" even when it's not a real error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 0.7
• Fix Version/s: Impala 1.0
• Assignee: Skye Wanderman-Milne
• Reporter: Lenni Kuff
• Created: 17/Apr/13 00:29
• Updated: 20/Dec/15 00:05
• Resolved: 25/Apr/13 00:01

Description

I can't remember why this was implemented this way but sequences files will sometimes spew this even when this is not an issue. I think it might be at the end of the file, some of the stuff is optional (e.g. sync hash marker). It's a little annoying we see this. We should try to only spew this if there is a real error.

Nong: I think the most common case that we should fix is when we are at the end of the file looking for the next sync marker/block of data. We should store the store the length of the file and not try to read past it.
Skye, this might be a good one to fix to get more familiar with the scanner code for avro.