+----------------------------------------------------+ | Explain | +----------------------------------------------------+ | STAGE DEPENDENCIES: | | Stage-20 is a root stage | | Stage-15 depends on stages: Stage-20 | | Stage-14 depends on stages: Stage-15 , consists of Stage-18, Stage-19, Stage-2 | | Stage-18 has a backup stage: Stage-2 | | Stage-12 depends on stages: Stage-18 | | Stage-11 depends on stages: Stage-2, Stage-12, Stage-13 , consists of Stage-16, Stage-17, Stage-3 | | Stage-16 has a backup stage: Stage-3 | | Stage-9 depends on stages: Stage-16 | | Stage-4 depends on stages: Stage-3, Stage-9, Stage-10 | | Stage-5 depends on stages: Stage-4 | | Stage-17 has a backup stage: Stage-3 | | Stage-10 depends on stages: Stage-17 | | Stage-3 | | Stage-19 has a backup stage: Stage-2 | | Stage-13 depends on stages: Stage-19 | | Stage-2 | | Stage-0 depends on stages: Stage-5 | | | | STAGE PLANS: | | Stage: Stage-20 | | Map Reduce Local Work | | Alias -> Map Local Tables: | | $hdt$_3:date_dim | | Fetch Operator | | limit: -1 | | Alias -> Map Local Operator Tree: | | $hdt$_3:date_dim | | TableScan | | alias: date_dim | | filterExpr: ((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null) (type: boolean) | | Statistics: Num rows: 73049 Data size: 1110368 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ((d_qoy = 2) and (d_year = 2000) and d_date_sk is not null) (type: boolean) | | Statistics: Num rows: 1 Data size: 15 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: d_date_sk (type: bigint) | | outputColumnNames: _col0 | | Statistics: Num rows: 1 Data size: 15 Basic stats: COMPLETE Column stats: NONE | | HashTable Sink Operator | | keys: | | 0 _col2 (type: bigint) | | 1 _col0 (type: bigint) | | | | Stage: Stage-15 | | Map Reduce | | Map Operator Tree: | | TableScan | | alias: catalog_sales | | filterExpr: (cs_bill_customer_sk is not null and cs_sold_date_sk is not null) (type: boolean) | | Statistics: Num rows: 1432777090 Data size: 33240428496 Basic stats: COMPLETE Column stats: PARTIAL | | Filter Operator | | predicate: cs_bill_customer_sk is not null (type: boolean) | | Statistics: Num rows: 1361138235 Data size: 31578407064 Basic stats: COMPLETE Column stats: PARTIAL | | Select Operator | | expressions: cs_bill_customer_sk (type: bigint), cs_sales_price (type: double), cs_sold_date_sk (type: bigint) | | outputColumnNames: _col0, _col1, _col2 | | Statistics: Num rows: 1361138235 Data size: 31578407064 Basic stats: COMPLETE Column stats: PARTIAL | | Map Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col2 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 1497252090 Data size: 34736248523 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | Execution mode: vectorized | | Local Work: | | Map Reduce Local Work | | | | Stage: Stage-14 | | Conditional Operator | | | | Stage: Stage-18 | | Map Reduce Local Work | | Alias -> Map Local Tables: | | $hdt$_1:customer | | Fetch Operator | | limit: -1 | | Alias -> Map Local Operator Tree: | | $hdt$_1:customer | | TableScan | | alias: customer | | filterExpr: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 12000000 Data size: 182400016 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: c_customer_sk (type: bigint), c_current_addr_sk (type: bigint) | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | HashTable Sink Operator | | keys: | +----------------------------------------------------+ | Explain | +----------------------------------------------------+ | 0 _col0 (type: bigint) | | 1 _col0 (type: bigint) | | | | Stage: Stage-12 | | Map Reduce | | Map Operator Tree: | | TableScan | | Map Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col0 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col7 | | Statistics: Num rows: 1646977334 Data size: 38209874203 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | Execution mode: vectorized | | Local Work: | | Map Reduce Local Work | | | | Stage: Stage-11 | | Conditional Operator | | | | Stage: Stage-16 | | Map Reduce Local Work | | Alias -> Map Local Tables: | | $hdt$_0:customer_address | | Fetch Operator | | limit: -1 | | Alias -> Map Local Operator Tree: | | $hdt$_0:customer_address | | TableScan | | alias: customer_address | | filterExpr: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 6000000 Data size: 2143200376 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: ca_address_sk (type: bigint), ca_state (type: string), ca_zip (type: string) | | outputColumnNames: _col0, _col1, _col2 | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | HashTable Sink Operator | | keys: | | 0 _col7 (type: bigint) | | 1 _col0 (type: bigint) | | | | Stage: Stage-9 | | Map Reduce | | Map Operator Tree: | | TableScan | | Map Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col7 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col9, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ((substr(_col10, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792') or (_col9) IN ('CA', 'WA', 'GA') or (_col1 > 500.0D)) (type: boolean) | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: _col1 (type: double), _col10 (type: string) | | outputColumnNames: _col1, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Group By Operator | | aggregations: sum(_col1) | | keys: _col10 (type: string) | | mode: hash | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | Execution mode: vectorized | | Local Work: | | Map Reduce Local Work | | | | Stage: Stage-4 | | Map Reduce | | Map Operator Tree: | | TableScan | | Reduce Output Operator | | key expressions: _col0 (type: string) | | sort order: + | | Map-reduce partition columns: _col0 (type: string) | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | TopN Hash Memory Usage: 0.04 | | value expressions: _col1 (type: double) | | Execution mode: vectorized | | Reduce Operator Tree: | +----------------------------------------------------+ | Explain | +----------------------------------------------------+ | Group By Operator | | aggregations: sum(VALUE._col0) | | keys: KEY._col0 (type: string) | | mode: mergepartial | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 905837553 Data size: 21015431267 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | | | Stage: Stage-5 | | Map Reduce | | Map Operator Tree: | | TableScan | | Reduce Output Operator | | key expressions: _col0 (type: string) | | sort order: + | | Statistics: Num rows: 905837553 Data size: 21015431267 Basic stats: COMPLETE Column stats: NONE | | TopN Hash Memory Usage: 0.04 | | value expressions: _col1 (type: double) | | Execution mode: vectorized | | Reduce Operator Tree: | | Select Operator | | expressions: KEY.reducesinkkey0 (type: string), VALUE._col0 (type: double) | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 905837553 Data size: 21015431267 Basic stats: COMPLETE Column stats: NONE | | Limit | | Number of rows: 100 | | Statistics: Num rows: 100 Data size: 2300 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | Statistics: Num rows: 100 Data size: 2300 Basic stats: COMPLETE Column stats: NONE | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | | | | Stage: Stage-17 | | Map Reduce Local Work | | Alias -> Map Local Tables: | | $INTNAME | | Fetch Operator | | limit: -1 | | Alias -> Map Local Operator Tree: | | $INTNAME | | TableScan | | HashTable Sink Operator | | keys: | | 0 _col7 (type: bigint) | | 1 _col0 (type: bigint) | | | | Stage: Stage-10 | | Map Reduce | | Map Operator Tree: | | TableScan | | alias: customer_address | | filterExpr: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 6000000 Data size: 2143200376 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: ca_address_sk (type: bigint), ca_state (type: string), ca_zip (type: string) | | outputColumnNames: _col0, _col1, _col2 | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | Map Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col7 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col9, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ((substr(_col10, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792') or (_col9) IN ('CA', 'WA', 'GA') or (_col1 > 500.0D)) (type: boolean) | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: _col1 (type: double), _col10 (type: string) | | outputColumnNames: _col1, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Group By Operator | | aggregations: sum(_col1) | | keys: _col10 (type: string) | | mode: hash | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | Execution mode: vectorized | | Local Work: | | Map Reduce Local Work | | | | Stage: Stage-3 | +----------------------------------------------------+ | Explain | +----------------------------------------------------+ | Map Reduce | | Map Operator Tree: | | TableScan | | Reduce Output Operator | | key expressions: _col7 (type: bigint) | | sort order: + | | Map-reduce partition columns: _col7 (type: bigint) | | Statistics: Num rows: 1646977334 Data size: 38209874203 Basic stats: COMPLETE Column stats: NONE | | value expressions: _col1 (type: double) | | TableScan | | alias: customer_address | | filterExpr: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 6000000 Data size: 2143200376 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ca_address_sk is not null (type: boolean) | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: ca_address_sk (type: bigint), ca_state (type: string), ca_zip (type: string) | | outputColumnNames: _col0, _col1, _col2 | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | Reduce Output Operator | | key expressions: _col0 (type: bigint) | | sort order: + | | Map-reduce partition columns: _col0 (type: bigint) | | Statistics: Num rows: 5700000 Data size: 2036040357 Basic stats: COMPLETE Column stats: NONE | | value expressions: _col1 (type: string), _col2 (type: string) | | Reduce Operator Tree: | | Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col7 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col9, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: ((substr(_col10, 1, 5)) IN ('85669', '86197', '88274', '83405', '86475', '85392', '85460', '80348', '81792') or (_col9) IN ('CA', 'WA', 'GA') or (_col1 > 500.0D)) (type: boolean) | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: _col1 (type: double), _col10 (type: string) | | outputColumnNames: _col1, _col10 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | Group By Operator | | aggregations: sum(_col1) | | keys: _col10 (type: string) | | mode: hash | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 1811675106 Data size: 42030862534 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | | | Stage: Stage-19 | | Map Reduce Local Work | | Alias -> Map Local Tables: | | $INTNAME | | Fetch Operator | | limit: -1 | | Alias -> Map Local Operator Tree: | | $INTNAME | | TableScan | | HashTable Sink Operator | | keys: | | 0 _col0 (type: bigint) | | 1 _col0 (type: bigint) | | | | Stage: Stage-13 | | Map Reduce | | Map Operator Tree: | | TableScan | | alias: customer | | filterExpr: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 12000000 Data size: 182400016 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: c_customer_sk (type: bigint), c_current_addr_sk (type: bigint) | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | Map Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col0 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col7 | | Statistics: Num rows: 1646977334 Data size: 38209874203 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | Execution mode: vectorized | | Local Work: | | Map Reduce Local Work | +----------------------------------------------------+ | Explain | +----------------------------------------------------+ | | | Stage: Stage-2 | | Map Reduce | | Map Operator Tree: | | TableScan | | Reduce Output Operator | | key expressions: _col0 (type: bigint) | | sort order: + | | Map-reduce partition columns: _col0 (type: bigint) | | Statistics: Num rows: 1497252090 Data size: 34736248523 Basic stats: COMPLETE Column stats: NONE | | value expressions: _col1 (type: double) | | TableScan | | alias: customer | | filterExpr: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 12000000 Data size: 182400016 Basic stats: COMPLETE Column stats: NONE | | Filter Operator | | predicate: (c_customer_sk is not null and c_current_addr_sk is not null) (type: boolean) | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | Select Operator | | expressions: c_customer_sk (type: bigint), c_current_addr_sk (type: bigint) | | outputColumnNames: _col0, _col1 | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | Reduce Output Operator | | key expressions: _col0 (type: bigint) | | sort order: + | | Map-reduce partition columns: _col0 (type: bigint) | | Statistics: Num rows: 10800000 Data size: 164160014 Basic stats: COMPLETE Column stats: NONE | | value expressions: _col1 (type: bigint) | | Reduce Operator Tree: | | Join Operator | | condition map: | | Inner Join 0 to 1 | | keys: | | 0 _col0 (type: bigint) | | 1 _col0 (type: bigint) | | outputColumnNames: _col1, _col7 | | Statistics: Num rows: 1646977334 Data size: 38209874203 Basic stats: COMPLETE Column stats: NONE | | File Output Operator | | compressed: false | | table: | | input format: org.apache.hadoop.mapred.SequenceFileInputFormat | | output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat | | serde: org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe | | | | Stage: Stage-0 | | Fetch Operator | | limit: 100 | | Processor Tree: | | ListSink | | | +----------------------------------------------------+