[DRILL-7730] Reduce overhead of web queries displayed in HTML

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.17.0
• Fix Version/s: 1.18.0
• Assignee: Paul Rogers
• Reporter: Paul Rogers
• Created: 04/May/20 19:25
• Updated: 27/May/20 21:07
• Resolved: 27/May/20 21:07

Description

Drill provides a web console to run queries. Query results appear as HTML pages. Drill buffers the query results in-memory to build the page. The current approach has two problems (in addition to the overhead of buffering):

• To move each batch from Screen to the REST client, we serialize all vectors into a single large buffer, then recreate the individual vectors.
• The code appends column names and metadata for each batch. For a multi-batch query, we end up with lists that contain many copies of the same data.

This change modifies the internal plumbing to transfer a record batch from Screen to REST without copying.