[DRILL-5290] Provide an option to build operator table once for built-in static functions and reuse it across queries.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0
• Fix Version/s: 1.10.0
• Labels: doc-impacting ready-to-commit
• Assignee: Padma Penumarthy
• Reporter: Padma Penumarthy
• Reviewer: Kunal Khatua
• Created: 22/Feb/17 18:51
• Updated: 29/Mar/17 21:57
• Resolved: 02/Mar/17 22:53

Description

Currently, DrillOperatorTable which contains standard SQL operators and functions and Drill User Defined Functions (UDFs) (built-in and dynamic) gets built for each query as part of creating QueryContext. This is an expensive operation ( ~30 msec to build) and allocates ~2M on heap for each query. For high throughput, low latency operational queries, we quickly run out of heap memory, causing JVM hangs. Build operator table once during startup for static built-in functions and save in DrillbitContext, so we can reuse it across queries.
Provide a system/session option to not use dynamic UDFs so we can use the operator table saved in DrillbitContext and avoid building each time.

Please note, changes are adding new option exec.udf.use_dynamic which needs to be documented.

Issue Links

• links to: GitHub Pull Request #757