[DRILL-740] Re-enable TPCH05 in unit tests after fixing underlying reason for timeout

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.4.0
• Assignee: Unassigned
• Reporter: Aman Sinha
• Created: 14/May/14 17:25
• Updated: 31/Jul/14 06:32
• Resolved: 09/Jun/14 05:34

Description

TPCH05 is currently marked Ignored since it started timing out after we enabled new plans with hash join and hash aggregate. The logs show that the query progresses until a point where some downstream fragment is waiting for data or end of stream and the query times out. More diagnostic information needs to be added to track exactly what each fragment is doing. We need to re-enable this test after fixing the underlying cause for timeout.

Here's the physical plan for query 05:

Drill Physical :
ScreenPrel: rowcount = 9.638871459960938E8, cumulative cost =

{8.14099147957207E10 rows, 1.3306858338906152E12 cpu, 0.0 io, 1.521937908014208E15 network}

, id = 17939
SingleMergeExchangePrel(sort0=[1 DESC]): rowcount = 9.638871459960938E8, cumulative cost =

{8.13135260811211E10 rows, 1.3305894451760156E12 cpu, 0.0 io, 1.521937908014208E15 network}

, id = 17938
SortPrel(sort0=[$1], dir0=[DESC]): rowcount = 9.638871459960938E8, cumulative cost =

{8.0349638935125E10 rows, 1.3228783480080469E12 cpu, 0.0 io, 1.514041744514208E15 network}

, id = 17937
HashToRandomExchangePrel(dist0=[[$1]]): rowcount = 9.638871459960938E8, cumulative cost =

{7.93857517891289E10 rows, 1.2078122418080312E12 cpu, 0.0 io, 1.514041744514208E15 network}

, id = 17936
HashAggPrel(group=[

{0}

], revenue=[SUM($1)]): rowcount = 9.638871459960938E8, cumulative cost =

{7.842186464313281E10 rows, 1.1923900474720938E12 cpu, 0.0 io, 1.506145581014208E15 network}

, id = 17935
HashToRandomExchangePrel(dist0=[[$0]]): rowcount = 9.638871459960938E9, cumulative cost =

{6.878299318317188E10 rows, 9.99612618272875E11 cpu, 0.0 io, 1.506145581014208E15 network}

, id = 17934
ProjectPrel(n_name=[$4], $f1=[*($17, -(1, $18))]): rowcount = 9.638871459960938E9, cumulative cost =

{5.914412172321094E10 rows, 8.453906749135E11 cpu, 0.0 io, 1.427183946014208E15 network}

, id = 17933
HashJoinPrel(condition=[=($20, $6)], joinType=[inner]): rowcount = 9.638871459960938E9, cumulative cost =

{4.950525026325E10 rows, 8.453906749055E11 cpu, 0.0 io, 1.427183946014208E15 network}

, id = 17932
HashToRandomExchangePrel(dist0=[[$6]]): rowcount = 3.75, cumulative cost =

{90.75 rows, 923.0 cpu, 0.0 io, 529408.0 network}

, id = 17920
HashJoinPrel(condition=[=($5, $2)], joinType=[inner]): rowcount = 3.75, cumulative cost =

{87.0 rows, 863.0 cpu, 0.0 io, 421888.0 network}

, id = 17919
HashToRandomExchangePrel(dist0=[[$2]]): rowcount = 1.0, cumulative cost =

{11.0 rows, 51.0 cpu, 0.0 io, 12288.0 network}

, id = 17917
FilterPrel(condition=[=(CAST($1):CHAR(6) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary", 'EUROPE')]): rowcount = 1.0, cumulative cost =

{10.0 rows, 35.0 cpu, 0.0 io, 0.0 network}

, id = 17916
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/region.parquet]], selectionRoot=/tpch/region.parquet, columns=[SchemaPath [`r_name`], SchemaPath [`r_regionkey`]]]]): rowcount = 5.0, cumulative cost =

{5.0 rows, 15.0 cpu, 0.0 io, 0.0 network}

, id = 16822
HashToRandomExchangePrel(dist0=[[$2]]): rowcount = 25.0, cumulative cost =

{50.0 rows, 500.0 cpu, 0.0 io, 409600.0 network}

, id = 17918
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/nation.parquet]], selectionRoot=/tpch/nation.parquet, columns=[SchemaPath [`n_name`], SchemaPath [`n_regionkey`], SchemaPath [`n_nationkey`]]]]): rowcount = 25.0, cumulative cost =

{25.0 rows, 100.0 cpu, 0.0 io, 0.0 network}

, id = 16833
HashToRandomExchangePrel(dist0=[[$13]]): rowcount = 1.7135771484375E10, cumulative cost =

{3.2369478684375E10 rows, 6.39761416125E11 cpu, 0.0 io, 1.4271839454848E15 network}

, id = 17931
HashJoinPrel(condition=[AND(=($2, $13), =($9, $14))], joinType=[inner]): rowcount = 1.7135771484375E10, cumulative cost =

{1.52337072E10 rows, 3.65589072375E11 cpu, 0.0 io, 3.743621454848E14 network}

, id = 17930
HashToRandomExchangePrel(dist0=[[$2]], dist1=[[$9]]): rowcount = 7.6158984375E9, cumulative cost =

{7.6178084625E9 rows, 1.82807504775E11 cpu, 0.0 io, 3.74362144256E14 network}

, id = 17928
HashJoinPrel(condition=[=($8, $6)], joinType=[inner]): rowcount = 7.6158984375E9, cumulative cost =

{1910025.0 rows, 2.5942275E7 cpu, 0.0 io, 2.5504256E10 network}

, id = 17927
HashToRandomExchangePrel(dist0=[[$6]]): rowcount = 843750.0, cumulative cost =

{885750.0 rows, 1.38315E7 cpu, 0.0 io, 2.4271872E10 network}

, id = 17925
HashJoinPrel(condition=[=($1, $5)], joinType=[inner]): rowcount = 843750.0, cumulative cost =

{42000.0 rows, 331500.0 cpu, 0.0 io, 7.9872E7 network}

, id = 17924
HashToRandomExchangePrel(dist0=[[$1]]): rowcount = 1500.0, cumulative cost =

{3000.0 rows, 28500.0 cpu, 0.0 io, 1.8432E7 network}

, id = 17921
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/customer.parquet]], selectionRoot=/tpch/customer.parquet, columns=[SchemaPath [`c_custkey`], SchemaPath [`c_nationkey`]]]]): rowcount = 1500.0, cumulative cost =

{1500.0 rows, 4500.0 cpu, 0.0 io, 0.0 network}

, id = 16829
HashToRandomExchangePrel(dist0=[[$2]]): rowcount = 3750.0, cumulative cost =

{33750.0 rows, 240000.0 cpu, 0.0 io, 6.144E7 network}

, id = 17923
FilterPrel(condition=[AND(>=($1, 1997-01-01), <($1, +(1997-01-01, 12)))]): rowcount = 3750.0, cumulative cost =

{30000.0 rows, 180000.0 cpu, 0.0 io, 0.0 network}

, id = 17922
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/orders.parquet]], selectionRoot=/tpch/orders.parquet, columns=[SchemaPath [`o_orderdate`], SchemaPath [`o_custkey`], SchemaPath [`o_orderkey`]]]]): rowcount = 15000.0, cumulative cost =

{15000.0 rows, 60000.0 cpu, 0.0 io, 0.0 network}

, id = 16879
HashToRandomExchangePrel(dist0=[[$1]]): rowcount = 60175.0, cumulative cost =

{120350.0 rows, 1263675.0 cpu, 0.0 io, 1.232384E9 network}

, id = 17926
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/lineitem.parquet]], selectionRoot=/tpch/lineitem.parquet, columns=[SchemaPath [`l_orderkey`], SchemaPath [`l_suppkey`], SchemaPath [`l_extendedprice`], SchemaPath [`l_discount`]]]]): rowcount = 60175.0, cumulative cost =

{60175.0 rows, 300875.0 cpu, 0.0 io, 0.0 network}

, id = 16839
HashToRandomExchangePrel(dist0=[[$1]], dist1=[[$2]]): rowcount = 100.0, cumulative cost =

{200.0 rows, 2700.0 cpu, 0.0 io, 1228800.0 network}

, id = 17929
ScanPrel(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=/tpch/supplier.parquet]], selectionRoot=/tpch/supplier.parquet, columns=[SchemaPath [`s_nationkey`], SchemaPath [`s_suppkey`]]]]): rowcount = 100.0, cumulative cost =

{100.0 rows, 300.0 cpu, 0.0 io, 0.0 network}

, id = 17110