[IMPALA-7053] Reorganise query options into groups

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Target Version: Product Backlog
• Epic Color: ghx-label-7
• Assignee: Bikramjeet Vig
• Reporter: Tim Armstrong
• Created: 21/May/18 17:21
• Updated: 24/Oct/18 17:32

Description

We have quite a lot of query options now and we're adding more for things like resource limits (e.g. IMPALA-6035). It's getting harder for users to understand the organisation and find relevant query options. We should consider grouping similar query options.

E.g. for this set of resource limits, we could reorganise in various ways:

• mem_limit -> resources.memory.per_node_limit
• buffer_pool_limit -> resources.memory.buffer_pool.per_node_limit
• thread_reservation_limit -> resources.threads.per_node_limit
• thread_reservation_aggregate_limit -> resources.threads.aggregate_limit
• exec_time_limit_s -> resources.wallclock.limit_s

We could do the conversion incrementally. It would probably make sense to agree on a top-level organisation up-front.

• planner - anything that controls planner decisions like join ordering, etc
• scheduler - anything that controls scheduler decisions (admission control could maybe be included here too)
• resources - resource management functionality (limits, etc)
• session - anything related to session management like timeouts
• exec - anything that changes query execution behaviour (e.g. codegen, batch sizes, runtime filters, etc)
• Probably a group for anything that changes the semantic behaviour of a query (e.g. decimal_v2, appx_count_distinct, strict_mode, abort_on_error).
• A group that controls read and write behaviour of file formats like compression, etc