Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5058 Improve concurrency of DDL/DML operations during catalog updates
  3. IMPALA-5538

Use explicit catalog versions for deleted objects in catalog updates

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 2.9.0
    • Impala 2.11.0
    • Catalog

    Description

      Catalog update topic updates include both additions and deletions. Even though added objects have a catalog version, deleted objects use the max catalog object version from the update message and not the catalog version that was assigned to the associated object when it was deleted. That works fine assuming the catalog updates do not have gaps (all updates between versions t1 and t2 are included). With the fix for IMPALA-5058, the semantics of what's included in the catalog updates will change, resulting in metadata inconsistencies if we don't assign the correct versions to deleted objects.

      Disclaimer: The above mentioned "versions" indicate the catalog versions set by the catalog server and not the statestore topic versions.

      Attachments

        Activity

          People

            dtsirogiannis Dimitris Tsirogiannis
            dtsirogiannis Dimitris Tsirogiannis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: