[DRILL-6374] Transitive Closure leads to TPCH Queries regressions and OOM when run concurrency test

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.0
• Component/s: Functions - Drill
• Labels: ready-to-commit

Environment

RHEL 7

• Assignee: Vitalii Diravka
• Reporter: Dechang Gu
• Reviewer: Aman Sinha
• Created: 01/May/18 22:23
• Updated: 23/May/18 18:21
• Resolved: 23/May/18 18:21

Description

Run TPCH regression test on Apache Drill 1.14.0 master commit 6fcaf4268eddcb09010b5d9c5dfb3b3be5c3f903 (DRILL-6173), most of the queries regressed.

In particular, TPC-H Query 9 takes about 4x time (36 sec vs 8.6 sec), comparing to that when run against the parent commit (9173308710c3decf8ff745493ad3e85ccdaf7c37).

Further in the concurrency test for the commit, with 48 clients each running 16 TPCH queries (so total 768 queries are executed) with planner.width.max_per_node=5, some queries hit OOM and caused 273 queries failed, while for the parent commit all the 768 queries completed successfully.

 

Profiles for TPCH_09 in the regression tests are uploaded:

• The failing commit  file name: TPCH_09_2_id_2517381b-1a61-3db5-40c3-4463bd421365.json,
• The parent commit file name: TPCH_09_2_id_2517497b-d4da-dab6-6124-abde5804a25f.json ).

Issue Links

• is duplicated by: DRILL-6384 TPC-H tests fail with OOM
• relates to: DRILL-6173 Support transitive closure during filter push down and partition pruning
• links to: GitHub Pull Request #1262