Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10147

Avoid getting a file handle for data cache hits

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 4.0.0
    • Impala 4.0.0
    • Backend
    • None

    Description

      When reading from the data cache, the DiskIo thread first gets a file handle, then it checks the data cache for a hit. If there is a cache hit, then the file handle is not actually used. It is only used if there is a cache miss. There is no real reason to have the file handle open for cache hits. It doesn't really serve any additional purpose, and it adds overhead to cache hits.

      For platforms that do not have the file handle cache, this can be a significant overhead.

      We should only open the file handle after we have checked the data cache and know that we need to read from regular storage.

      Attachments

        Issue Links

          Activity

            People

              rizaon Riza Suminto
              joemcdonnell Joe McDonnell
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: