[DRILL-3900] OOM with Hive native scan enabled on TPCH-100 parquet, query 05.q

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.0
• Component/s: Functions - Hive
• Assignee: Unassigned
• Reporter: Chun Chang
• Created: 06/Oct/15 17:28
• Updated: 04/Jan/18 15:10

Description

TPCH-100 parquet dataset. Configure Hive 1.0 pointing to the parquet files as external tables. Enable Hive native scan.

alter system set `store.hive.optimize_scan_with_native_readers`=true;

Run TPCH query 05 through Hive, drillbit runs out of memory. Same query goes through dfs completes successfully. (Disable hive native scan, drill also runs out of memory through hive.)

We expect with hive native scan turned on, query should finish.

Issue Links

• blocks: DRILL-4309 Make this option store.hive.optimize_scan_with_native_readers=true default