Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12906

Incorporate run time scan range information into the tuple cache key

    XMLWordPrintableJSON

Details

    • Task
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • Impala 4.4.0
    • None
    • Backend, Frontend
    • None

    Description

      The cache key for tuple caching currently doesn't incorporate information about the scan ranges for the tables that it scans. This is important for detecting changes in the table and having different cache keys for different fragment instances that are assigned different scan ranges.

      To make this deterministic for mt_dop, we need mt_dop to assign scan ranges deterministically to individual fragment instances rather than using the shared queue introduced inĀ IMPALA-9655.

      One way to implement this is to collect information about the scan nodes that feed into the tuple cache and pass that information over to the tuple cache node. At runtime, it can hash the scan ranges assigned to those scan nodes and incorporate that into the cache key.

      Attachments

        Activity

          People

            joemcdonnell Joe McDonnell
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: