Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1570

DROP / COMPUTE incremental stats with dynamic partition specs

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      COMPUTE INCREMENTAL STATS, and its counterpart DROP INCREMENTAL STATS can both take PARTITION ... clauses to specify the precise partition to act on. If a small set of partitions need updating (or if the user wants to batch incremental stat computations in clumps), it would be good to allow dynamic partition specs, and only drop or update those partitions matched by the looser specification.

      For example COMPUTE INCREMENTAL STATS tbl PARTITION(year=2009, month) would update all months in 2009 that were missing incremental stats.

      For maximum benefit, we could add a set of constraints:

      COMPUTE INCREMENTAL STATS tbl PARTITION(year, month) WHERE year > 2009 and month=10

      would update all the October partitions since 2010 inclusive.

      We already have logic to do partition pruning in the frontend - this is the reverse. COMPUTE INCREMENTAL STATS already is able to work on a subset of partitions, so the work here should be substantially in the frontend.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amosbird Amos Bird
            henryr Henry Robinson
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment