Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8341

Data cache for remote reads

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 3.2.0
    • Impala 3.3.0
    • Backend
    • None
    • ghx-label-4

    Description

      When running in public cloud (e.g. AWS with S3) or in certain private cloud settings (e.g. data stored in object store), the computation and storage are no longer co-located. This breaks the typical pattern in which Impala query fragment instances are scheduled at where the data is located. In this setting, the network bandwidth requirement of both the nics and the top of rack switches will go up quite a lot as the network traffic includes the data fetch in addition to the shuffling exchange traffic of intermediate results.

      To mitigate the pressure on the network, one can build a storage backed cache at the compute nodes to cache the working set. With deterministic scan range scheduling, each compute node should hold non-overlapping partitions of the data set. 

      An initial prototype of the cache was posted here: https://gerrit.cloudera.org/#/c/12683/ but it probably can benefit from a better eviction algorithm (e.g. LRU instead of FIFO) and better locking (e.g. not holding the lock while doing IO).

      Attachments

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-8562 Data cache should skip scan range with mtime == -1

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-8496 test_data_cache.py is flaky

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-8512 Data cache tests failing on older CentOS 6 versions

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-4568 Cache Parquet footer cache to speedup scans & predicate evaluation against Min/Max indexes

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-8691 Query hint for disabling data caching

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-9819 Separate data cache and HDFS scan node runtime profile metrics

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-8542 Access trace collection for data cache

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-8690 Better eviction algorithm for data cache

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-6057 Cache Remote Reads

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. IMPALA-8495 Impala Doc: Document Data Read Cache

          • Major - Major loss of function.
          • Closed

          Activity

            People

              kwho Michael Ho
              kwho Michael Ho
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: