Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8877

CatalogException during stress test: Table <foo> modified while operation was in progress

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.3.0
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:
    • Epic Color:
      ghx-label-4

      Description

      This was hit while running the stress tests to get a baseline on a deployed cluster.

      /* Mem: 12850 MB. Coordinator: quasar-mzmnbe-6.vpc.cloudera.com. */
      COMPUTE STATS catalog_sales

      Query (id=924a50178a5a6146:29d58a7300000000)
        Summary
          Session ID: 5543fb9029e2b71f:f446381b1f59ed81
          Session Type: HIVESERVER2
          HiveServer2 Protocol Version: V6
          Start Time: 2019-08-19 01:26:07.292866000
          End Time: 2019-08-19 01:26:27.248053000
          Query Type: DDL
          Query State: EXCEPTION
          Query Status: CatalogException: Table 'tpcds_300_decimal_parquet.catalog_sales' was modified while operation was in progress, aborting execution.
          Impala Version: impalad version 3.3.0-SNAPSHOT RELEASE (build df3e7c051e2641524fc53a0cd07c2a14decd55f7)
          User: systest@VPC.CLOUDERA.COM
          Connected User: systest@VPC.CLOUDERA.COM
          Delegated User: 
          Network Address: ::ffff:10.65.6.19:39174
          Default Db: tpcds_300_decimal_parquet
          Sql Statement: /* Mem: 12850 MB. Coordinator: quasar-mzmnbe-6.vpc.cloudera.com. */
      COMPUTE STATS catalog_sales
          Coordinator: quasar-mzmnbe-6.vpc.cloudera.com:22000
          Query Options (set by configuration): ABORT_ON_ERROR=1,MEM_LIMIT=13474201600,MT_DOP=4,EXEC_TIME_LIMIT_S=2147483647,TIMEZONE=America/Los_Angeles,DEFAULT_FILE_FORMAT=4,DEFAULT_TRANSACTIONAL_TYPE=1
          Query Options (set by configuration and planner): ABORT_ON_ERROR=1,MEM_LIMIT=13474201600,MT_DOP=4,EXEC_TIME_LIMIT_S=2147483647,TIMEZONE=America/Los_Angeles,DEFAULT_FILE_FORMAT=4,DEFAULT_TRANSACTIONAL_TYPE=1
          DDL Type: COMPUTE_STATS
          Query Compilation
            Metadata of all 1 tables cached: 5.62s (5622372318)
            Analysis finished: 5.62s (5622560027)
            Authorization finished (noop): 5.62s (5622568284)
            Retried query planning due to inconsistent metadata 7 of 40 times: Catalog object TCatalogObject(type:TABLE, catalog_version:94204, table:TTable(db_name:tpcds_300_decimal_parquet, tbl_name:catalog_sales)) changed version between accesses.: 5.95s (5949859598)
            Planning finished: 5.95s (5949861145)
          Query Timeline
            Query submitted: 0ns (0)
            Planning finished: 5.95s (5950024020)
            Child queries finished: 17.85s (17849072057)
            Rows available: 19.82s (19825080035)
            Unregister query: 19.95s (19955080560)
          Frontend
            - CatalogFetch.ColumnStats.Misses: 34 (34)
            - CatalogFetch.ColumnStats.Requests: 34 (34)
            - CatalogFetch.ColumnStats.Time: 0 (0)
            - CatalogFetch.Config.Hits: 1 (1)
            - CatalogFetch.Config.Requests: 1 (1)
            - CatalogFetch.Config.Time: 0 (0)
            - CatalogFetch.DatabaseList.Hits: 8 (8)
            - CatalogFetch.DatabaseList.Requests: 8 (8)
            - CatalogFetch.DatabaseList.Time: 0 (0)
            - CatalogFetch.PartitionLists.Misses: 1 (1)
            - CatalogFetch.PartitionLists.Requests: 1 (1)
            - CatalogFetch.PartitionLists.Time: 7 (7)
            - CatalogFetch.Partitions.Hits: 1837 (1837)
            - CatalogFetch.Partitions.Misses: 1837 (1837)
            - CatalogFetch.Partitions.Requests: 3674 (3674)
            - CatalogFetch.Partitions.Time: 325 (325)
            - CatalogFetch.RPCs.Bytes: 4.7 MiB (4936030)
            - CatalogFetch.RPCs.Requests: 22 (22)
            - CatalogFetch.RPCs.Time: 343 (343)
            - CatalogFetch.TableNames.Hits: 4 (4)
            - CatalogFetch.TableNames.Misses: 4 (4)
            - CatalogFetch.TableNames.Requests: 8 (8)
            - CatalogFetch.TableNames.Time: 0 (0)
            - CatalogFetch.Tables.Misses: 8 (8)
            - CatalogFetch.Tables.Requests: 8 (8)
            - CatalogFetch.Tables.Time: 74 (74)
            - InactiveTotalTime: 0ns (0)
            - TotalTime: 0ns (0)
        ImpalaServer
          - CatalogOpExecTimer: 1.97s (1972007962)
          - ClientFetchWaitTimer: 0ns (0)
          - InactiveTotalTime: 0ns (0)
          - RowMaterializationTimer: 0ns (0)
          - TotalTime: 0ns (0)
        Child Queries
          Table Stats Query (id=db4821e4aa5bb04d:d4a5ae4500000000)
          Column Stats Query (id=0444367557e3496d:f943511100000000)
      

        Attachments

        1. catalogd.INFO.tar.gz
          6.12 MB
          David Knupp
        2. impalad.INFO.tar.gz
          5.87 MB
          David Knupp

          Activity

            People

            • Assignee:
              vihangk1 Vihang Karajgaonkar
              Reporter:
              dknupp David Knupp
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: