[DRILL-4678] Tune metadata by generating a dispatcher at runtime

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.11.0
• Component/s: Query Planning & Optimization

Environment

4 node cluster CentOS

• Assignee: Serhii Harnyk
• Reporter: Khurram Faraaz
• Reviewer: Jinfeng Ni
• Created: 16/May/16 11:59
• Updated: 11/Jul/17 22:29
• Resolved: 30/Mar/17 00:33

Description

Below query hangs

2016-05-16 10:33:57,506 [28c65de9-9f67-dadb-5e4e-e1a12f8dda49:foreman] INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id 28c65de9-9f67-dadb-5e4e-e1a12f8dda49: SELECT DISTINCT dt FROM (
VALUES(CAST('1964-03-07' AS DATE)),
      (CAST('2002-03-04' AS DATE)),
      (CAST('1966-09-04' AS DATE)),
      (CAST('1993-08-18' AS DATE)),
      (CAST('1970-06-11' AS DATE)),
      (CAST('1970-06-11' AS DATE)),
      (CAST('1970-06-11' AS DATE)),
      (CAST('1970-06-11' AS DATE)),
      (CAST('1970-06-11' AS DATE)),
      (CAST('1959-10-23' AS DATE)),
      (CAST('1992-01-14' AS DATE)),
      (CAST('1994-07-24' AS DATE)),
      (CAST('1979-11-25' AS DATE)),
      (CAST('1945-01-14' AS DATE)),
      (CAST('1982-07-25' AS DATE)),
      (CAST('1966-09-06' AS DATE)),
      (CAST('1989-05-01' AS DATE)),
      (CAST('1996-03-08' AS DATE)),
      (CAST('1998-08-19' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
      (CAST('2013-08-13' AS DATE)),
(CAST('1999-07-20' AS DATE)),
    (CAST('1962-07-03' AS DATE)),
      (CAST('2011-08-17' AS DATE)),
      (CAST('2011-05-16' AS DATE)),
      (CAST('1946-05-08' AS DATE)),
      (CAST('1994-02-13' AS DATE)),
      (CAST('1978-08-09' AS DATE)),
      (CAST('1978-08-09' AS DATE)),
      (CAST('1978-08-09' AS DATE)),
      (CAST('1978-08-09' AS DATE)),
      (CAST('1958-02-06' AS DATE)),
      (CAST('2012-06-11' AS DATE)),
      (CAST('2012-06-11' AS DATE)),
      (CAST('2012-06-11' AS DATE)),
      (CAST('2012-06-11' AS DATE)),
      (CAST('1998-03-26' AS DATE)),
      (CAST('1996-11-04' AS DATE)),
      (CAST('1953-09-25' AS DATE)),
      (CAST('2003-06-17' AS DATE)),
      (CAST('2003-06-17' AS DATE)),
      (CAST('2003-06-17' AS DATE)),
      (CAST('2003-06-17' AS DATE)),
      (CAST('2003-06-17' AS DATE)),
      (CAST('1980-07-05' AS DATE)),
      (CAST('1982-06-15' AS DATE)),
      (CAST('1951-05-16' AS DATE)))
tbl(dt)

Details from Web UI Profile tab, please note that the query is still in STARTING state

Running Queries
Time	User	Query	State	Foreman
05/16/2016 10:33:57	
mapr
 SELECT DISTINCT dt FROM ( VALUES(CAST('1964-03-07' AS DATE)), (CAST('2002-03-04' AS DATE)), (CAST('1966-09-04' AS DATE)), (CAST('199
STARTING
centos-01.qa.lab

There is no other useful information in drillbit.log. jstack output is attached here for your reference.

The same query works fine on Postgres 9.3

Issue Links

• is related to: DRILL-5468 TPCH Q18 regressed ~3x due to execution plan changes
• is related to: DRILL-4347 Planning time for query64 from TPCDS test suite has increased 10 times compared to 1.4 release
• relates to: CALCITE-1169 Memory leak caused by CALCITE-604 and CALCITE-1147
• links to: GitHub Pull Request #793