Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10545

Tune data_cache_write_concurrency based on the type of IO device

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 4.0.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
    • Epic Color:
      ghx-label-14

      Description

      The data cache limits concurrency writes to the cache to avoid overwhelming the underlying IO device. This is controlled by the data_cache_write_concurrency flags and defaults to 1. For SSDs, we should be able to increase this to allow more concurrent writes to the data cache. This would allow the data cache to warm up faster and stay more up to date.

      One option is to detect the underlying IO device (similar to how we do this for other parts of Disk IO Mgr) and tune this parameter higher for SSDs.

        Attachments

          Activity

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar
              Reporter:
              joemcdonnell Joe McDonnell
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: