Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10663

Coordinator might observe stale metadata in local catalog mode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Catalog, Frontend
    • Labels:
      None
    • Epic Color:
      ghx-label-12

      Description

      Coordinator might observe stale metadata in local catalog mode, even if the DDL operation was executed on the same coordinator.

      The issue is reproducible in the dev environment using local catalog mode:

      bin/start-impala-cluster.py --impalad_args --use_local_catalog=true --catalogd_args --catalog_topic_mode=minimal --catalogd_args --hms_event_polling_interval_s=1
      
      CREATE TABLE doub (d double);
      

      Renaming the table back and forth and querying from it reproduces the issue:

      bin/impala-shell.sh -q "alter table doub rename to doub_tmp; select * from doub_tmp; alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select * from doub;"

      ...
      ... then, after a while ...
      ...
      Query: alter table doub rename to doub_tmp
      +--------------------------+
      | summary                  |
      +--------------------------+
      | Renaming was successful. |
      +--------------------------+
      Fetched 1 row(s) in 0.02s
      Query: select * from doub_tmp
      Query submitted at: 2021-04-20 14:49:52 (Coordinator: http://localhost:25000)
      ERROR: AnalysisException: Could not resolve table reference: 'doub_tmp'
      
      Could not execute command: select * from doub_tmp
      

      Then, if I execute the same SELECT a bit later:

      $ bin/impala-shell.sh -q "select * from doub_tmp"
      
      Fetched 0 row(s) in 0.12s
      

      I think if the DDL operation succeeds, then the following operations on the same coordinator should see its effect.

      The current behavior makes tests flaky. The workaround is to set sync_ddl=true, but it makes these operations very slow.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              boroknagyz Zoltán Borók-Nagy
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: