[DRILL-3512] json queries fails on unexpected EOL

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Fix Version/s: Future
• Component/s: Storage - JSON
• Assignee: Unassigned
• Reporter: Stefán Baxter
• Created: 17/Jul/15 23:09
• Updated: 01/Oct/15 16:36

Description

querying directories or files that are open by some writer is a create way to keep up with data (csv/tsv) being logged out by servers, processes etc.

We did a small test where we appended a copy of our message stream (1k msg./sec.) to a json file and queried it with Drill. This worked just fine but every once in a while we would get Unexpected End of Line errors as the writer was appending to the file as Drill was reading it.

It would be great if an incomplete entry at the end of the file would simply be ignored.