Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6948

Coordinators don't detect the deletion of tables that occurred outside of impala after catalog restart

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 3.0, Impala 2.12.0
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:
    • Epic Color:
      ghx-label-1

      Description

      Upon catalog restart the coordinators detect this event and request a full topic update from the statestore. In certain cases, the topic update protocol executed between the statestore and the catalog fails to detect catalog objects that were deleted from the Metastore externally (e.g. via HIVE), thus causing these objects to show up again in each coordinator's catalog cache. The end result is that the catalog server and the coordinator's cache are out of sync and in some cases the only solution is to restart both the catalog and the statestore. 

      The following sequence can reproduce this issue:

      impala> create table lala(int a);
      bash> kill -9 `pidof catalogd`
      hive> drop table lala;
      bash> restart catalogd 
      impala> show tables;
      --- lala shows up in the list of tables;

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dtsirogiannis Dimitris Tsirogiannis
                Reporter:
                dtsirogiannis Dimitris Tsirogiannis
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: