Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7453

Intern HdfsStorageDescriptors

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.1.0
    • Catalog
    • None
    • ghx-label-5

    Description

      Every partition currently has an HdfsStorageDescriptor attached. In most cases, the number of unique storage descriptors in a warehouse is pretty low (most partitions use the same escaping, file formats, etc). For example, in the functional test data load, we only have 24 unique SDs across ~10k partitions. Each object takes 32 bytes (with compressed oops) or 40 (without). So, we can get some small memory/object-count savings by interning these.

      Attachments

        Activity

          People

            Unassigned Unassigned
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: