Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9435

Usability enhancements for the data cache access trace

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.4.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:
      None
    • Target Version:
    • Epic Color:
      ghx-label-4

      Description

      In IMPALA-8542, we added an access trace to the data cache. This outputs a JSON entry with information about each hit/miss. Currently it is controlled by the data_cache_enable_tracing startup parameter, and it will trace all accesses to a single file. There are a few enhancements that would make this easier to enable:

      1. Limit the number of access trace entries stored on disk to avoid unlimited disk usage. This can be done by switching to use be/src/util/simple-logger.h rather than a single file. The number of retained entries should be configurable.
      2. Implement a way to trace a subset of accesses (e.g. 5%)
      3. Optionally provide a way to start/stop logging without restart (e.g. via the WebUI)

        Attachments

          Activity

            People

            • Assignee:
              joemcdonnell Joe McDonnell
              Reporter:
              joemcdonnell Joe McDonnell
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: