[DRILL-6830] Hook.REL_BUILDER_SIMPLIFY handler didn't removed cause performance degression

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.16.0
• Component/s: Query Planning & Optimization
• Assignee: shuifeng lu
• Reporter: shuifeng lu
• Reviewer: Vova Vysotskyi
• Created: 06/Nov/18 09:01
• Updated: 02/Oct/19 18:38
• Resolved: 02/Feb/19 11:23

Description

Planning performance degression has been observed that the duration of planning increased from 30ms to 160ms after running drill a long period of time(say a month).

RelBuilder.simplify never becomes true if Hook.REL_BUILDER_SIMPLIFY handlers are not removed.

Here is some clue (after running 40 days):

Hook.get takes 8ms per-invocation, it may be called serveral times per query.
---[8.816063ms] org.apache.calcite.tools.RelBuilder:<init>()
  +---[0.020218ms] java.util.ArrayDeque:<init>()
  +---[0.018493ms] java.lang.Boolean:valueOf()
  +---[8.341566ms] org.apache.calcite.runtime.Hook:get()
  +---[0.008489ms] java.lang.Boolean:booleanValue()
  +---[min=5.21E-4ms,max=0.015832ms,total=0.025233ms,count=12] org.apache.calcite.plan.Context:unwrap()
  +---[min=3.83E-4ms,max=0.009494ms,total=0.014516ms,count=13] org.apache.calcite.util.Util:first()
  +---[0.006892ms] org.apache.calcite.plan.RelOptCluster:getPlanner()
  +---[0.009104ms] org.apache.calcite.plan.RelOptPlanner:getExecutor()
  +---[min=4.8E-4ms,max=0.002277ms,total=0.002757ms,count=2] org.apache.calcite.plan.RelOptCluster:getRexBuilder()
  ---[min=4.91E-4ms,max=0.004586ms,total=0.005077ms,count=2] org.apache.calcite.rex.RexSimplify:<init>()

The top instances in JVM
num   #instances     #bytes           class name
----------------------------------------------
1:       116333      116250440     [B
2:       890126      105084536    [C
3:       338062        37415944    [Ljava.lang.Object;
4:     1715004        27440064    org.apache.calcite.runtime.Hook$4
5:      803909         19293816    java.lang.String

 

Issue Links

• is part of: DRILL-6862 Update Calcite to 1.18.0
• links to: GitHub Pull Request #1524