Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5648

Count star optimisation regressed Parquet memory estimate accuracy

    XMLWordPrintableJSON

Details

    Description

      The Parquet memory estimate is based on the number of columns scanned in the file. Before IMPALA-5036, count queries correctly counted zero columns read because it was a metadata-only scan. The count-star optimisation is also a metadata-only scan but the memory estimate is based on the column actually being scanned.

      This regression is apparent in the planner test changes:
      https://gerrit.cloudera.org/#/c/6812/12/testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test

      Attachments

        Issue Links

          Activity

            People

              tarmstrong Tim Armstrong
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: