Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10737

Optimize Iceberg metadata handling

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Frontend
    • Labels:

      Description

      Currently we re-read Iceberg table metadata in several cases.

      We should rather keep it in memory and use it when possible.

      Also, when refreshing a table we should use Iceberg's refresh() API to avoid unnecessary re-reads of manifest files:

      https://github.com/apache/iceberg/blob/282b6f9f1cae8d4fd5ff7c73de513ca91f01fddc/core/src/main/java/org/apache/iceberg/TableOperations.java#L45

       

        Attachments

          Activity

            People

            • Assignee:
              boroknagyz Zoltán Borók-Nagy
              Reporter:
              boroknagyz Zoltán Borók-Nagy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: