Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3900

OOM with Hive native scan enabled on TPCH-100 parquet, query 05.q

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.2.0
    • Fix Version/s: None
    • Component/s: Functions - Hive
    • Labels:
      None

      Description

      TPCH-100 parquet dataset. Configure Hive 1.0 pointing to the parquet files as external tables. Enable Hive native scan.

      alter system set `store.hive.optimize_scan_with_native_readers`=true;

      Run TPCH query 05 through Hive, drillbit runs out of memory. Same query goes through dfs completes successfully. (Disable hive native scan, drill also runs out of memory through hive.)

      We expect with hive native scan turned on, query should finish.

        Attachments

          Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. DRILL-4309 Make this option store.hive.optimize_scan_with_native_readers=true default

          • Major - Major loss of function.
          • Open

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                cchang@maprtech.com Chun Chang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: