[DRILL-3543] Add stats for external sort to a query profile

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.4.0
• Component/s: Execution - Relational Operators
• Labels: documentation usability
• Assignee: Abdel Hakim Deneche
• Reporter: Victoria Markman
• Reviewer: Victoria Markman
• Created: 23/Jul/15 00:57
• Updated: 16/Dec/15 00:23
• Resolved: 23/Nov/15 23:21

Description

The only indication if sort spilled to disk today is info from the drillbit.log.
It would be great if this information was displayed in the query profile.

015-07-22 23:47:29,907 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO  o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/92
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO  o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO  o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] WARN  o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 7 batch groups. Current allocated memory: 11566787