Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8301

Eliminate need for SYNC_DDL in local catalog mode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • Impala 4.0.0
    • Catalog
    • ghx-label-6

    Description

      In the following scenario looks like the INSERT on coordinator 2 is gated behind a long-running DDL on coordinator 1. That scenario still requires SYNC_DDL even in metadata v2. We want to change the metadata handling design so that coordinator 3 does not have to wait for the long-running DDL to complete – it would render the correct result on the target table t1 reference.

      Step1. coordinator 1
      ##say, the following compute stats runs about 100 seconds
      compute stats tao_ddl_contention;

      [steps 2 and 3 are performed while COMPUTE STATS is running]

      Step2. coordinator 2
      create another new table.

      create table t1(c1 int);

      insert into t1 select 1 ;

      select * from t1;
      [can see the inserted rows]

      Step3. coordinator 3
      query the newly inserted rows in t1 while the COMPUTE STATS is still running:

      select * from t1;
      [see no rows]

      ##the query in [Step3] won't show the row inserted by step2 until the first step "compute stats" completed, unless:

      • SYNC_DDL is set before the INSERT on coordinator 2, or
      • when the step1 compute stats is completed, or
      • you can see the data from this impala session, or
      • a manual refresh of the t1 table.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bgitline Boris Gitline
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: