Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Column pruning will not work when we have an Aggregate on top of a DruidQuery and the aggregate cannot be pushed to druid. (one such case is when it is count on a metric).
To fix this, we can introduce a new Rule to extract a Project from the aggregate and push that to DruidQuery before pushing the aggregate.
INFO : Executing command(queryId=hive_20171020180303_09fd3ab2-6e4a-42a1-9e85-4bca0e13460b): explain SELECT COUNT(`__time`)
FROM tpcds_denormalized_druid_table_300M
WHERE `__time` >= '1999-11-01 00:00:00'
AND `__time` <= '1999-11-10 00:00:00'
AND `__time` < '1999-11-05 00:00:00'
INFO : Starting task [Stage-3:EXPLAIN] in serial mode
INFO : Resetting the caller context to HIVE_SSN_ID:a5e1f82e-6d6c-405c-a6da-0d74f2248603
INFO : Completed executing command(queryId=hive_20171020180303_09fd3ab2-6e4a-42a1-9e85-4bca0e13460b); Time taken: 0.011 seconds
INFO : OK
tpcds_real_bin_partitioned_orc_1000@tpcds_denormalized_druid_table_300m,tpcds_denormalized_druid_table_300m,Tbl:COMPLETE,Col:NONE,Output:["__time"],properties:{"druid.query.json":"{\"queryType\":\"select\",\"dataSource\":\"tpcds_real_bin_partitioned_orc_1000.tpcds_denormalized_druid_table_300M\",\"descending\":false,\"intervals\":[\"1999-11-01T00:00:00.000/1999-11-05T00:00:00.000\"],\"dimensions\":[\"i_item_id\",\"i_rec_start_date\",\"i_rec_end_date\",\"i_item_desc\",\"i_brand_id\",\"i_brand\",\"i_class_id\",\"i_class\",\"i_category_id\",\"i_category\",\"i_manufact_id\",\"i_manufact\",\"i_size\",\"i_formulation\",\"i_color\",\"i_units\",\"i_container\",\"i_manager_id\",\"i_product_name\",\"c_customer_id\",\"c_salutation\",\"c_first_name\",\"c_last_name\",\"c_preferred_cust_flag\",\"c_birth_day\",\"c_birth_month\",\"c_birth_year\",\"c_birth_country\",\"c_login\",\"c_email_address\",\"c_last_review_date\",\"ca_address_id\",\"ca_street_number\",\"ca_street_name\",\"ca_street_type\",\"ca_suite_number\",\"ca_city\",\"ca_county\",\"ca_state\",\"ca_zip\",\"ca_country\",\"ca_gmt_offset\",\"s_rec_end_date\",\"s_store_name\",\"s_hours\",\"s_manager\",\"s_market_id\",\"s_geography_class\",\"s_market_desc\",\"s_market_manager\",\"s_division_id\",\"s_division_name\",\"s_company_id\",\"s_company_name\",\"s_street_number\",\"s_street_name\",\"s_street_type\",\"s_suite_number\",\"s_city\",\"s_county\",\"s_state\",\"s_zip\",\"s_country\",\"s_gmt_offset\"],\"metrics\":[\"ss_ticket_number\",\"ss_quantity\",\"ss_wholesale_cost\",\"ss_list_price\",\"ss_sales_price\",\"ss_ext_discount_amt\",\"ss_ext_sales_price\",\"ss_ext_wholesale_cost\",\"ss_ext_list_price\",\"ss_ext_tax\",\"ss_coupon_amt\",\"ss_net_paid\",\"ss_net_paid_inc_tax\",\"ss_net_profit\",\"i_current_price\",\"i_wholesale_cost\",\"s_number_employees\",\"s_floor_space\",\"s_tax_precentage\"],\"granularity\":\"all\",\"pagingSpec\":{\"threshold\":16384,\"fromNext\":true},\"context\":{\"druid.query.fetch\":false}}","druid.query.type":"select"} |