Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1570

DROP / COMPUTE incremental stats with dynamic partition specs

    Details

      Description

      COMPUTE INCREMENTAL STATS, and its counterpart DROP INCREMENTAL STATS can both take PARTITION ... clauses to specify the precise partition to act on. If a small set of partitions need updating (or if the user wants to batch incremental stat computations in clumps), it would be good to allow dynamic partition specs, and only drop or update those partitions matched by the looser specification.

      For example COMPUTE INCREMENTAL STATS tbl PARTITION(year=2009, month) would update all months in 2009 that were missing incremental stats.

      For maximum benefit, we could add a set of constraints:

      COMPUTE INCREMENTAL STATS tbl PARTITION(year, month) WHERE year > 2009 and month=10

      would update all the October partitions since 2010 inclusive.

      We already have logic to do partition pruning in the frontend - this is the reverse. COMPUTE INCREMENTAL STATS already is able to work on a subset of partitions, so the work here should be substantially in the frontend.

        Attachments

          Activity

            People

            • Assignee:
              amosbird Amos Bird
              Reporter:
              henryr Henry Robinson
            • Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: