Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
While testing fixes for the parquet nullable support I ran into an issue with unexpected results. I was selecting several columns out of file parquet file, which supports project pushdown. Currently the planner still includes a project operation after the scan in this case (to properly modify schema in the case of array indexing, project pushdown into scans is currently not supposed to be changing structure). I pulled the physical plan from the query and ran it without the extra project (as I was not selecting any array values) and got the expected results.
Here is the query I ran, the file is too large to attach so you can e-mail me to get a copy of it.
select pig_schema,ss_sold_date_sk,ss_item_sk,ss_cdemo_sk,ss_addr_sk, ss_hdemo_sk from store_sales