Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8995

TestAdmissionController.test_statestore_outage seems flaky

    XMLWordPrintableJSON

Details

    • ghx-label-14

    Description

      It appears the test failed because an expected query admission result didn't happen during test statestore outage.

      Error Message
      AssertionError: Query (id=a644e4b355da6009:434111e500000000):     DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance.     Summary:       Session ID: 3e4e788067a3a4a3:cf244f82bb27e895       Session Type: BEESWAX       Start Time: 2019-10-01 01:12:50.075066000       End Time:        Query Type: QUERY       Query State: EXCEPTION       Query Status: Admission for query exceeded timeout 60000ms in pool default-pool. Queued reason: queue is not empty (size 2); queued queries are executed first. Warning: admission control information from statestore is stale: 1s258ms since last update was received.       Impala Version: impalad version 3.4.0-SNAPSHOT DEBUG (build b1d0659fe69bc43508735de17d7a3b4626b7138a)       User: jenkins       Connected User: jenkins       Delegated User:        Network Address: 127.0.0.1:43703       Default Db: default       Sql Statement: select sleep(100)       Coordinator: shared-centos64-ec2-m2-4xlarge-ondemand-0691.vpc.cloudera.com:22000       Query Options (set by configuration): TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=custom_cluster/test_admission_controller.py::TestAdmissionController::()::test_statestore_outage       Query Options (set by configuration and planner): NUM_NODES=1,NUM_SCANNER_THREADS=1,RUNTIME_FILTER_MODE=0,MT_DOP=0,TIMEZONE=America/Los_Angeles,CLIENT_IDENTIFIER=custom_cluster/test_admission_controller.py::TestAdmissionController::()::test_statestore_outage       Plan:    ----------------   Max Per-Host Resource Reservation: Memory=0B Threads=1   Per-Host Resource Estimates: Memory=10MB   Codegen disabled by planner   Analyzed query: SELECT sleep(CAST(100 AS INT))      F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1   |  Per-Host Resources: mem-estimate=0B mem-reservation=0B thread-reservation=1   PLAN-ROOT SINK   |  output exprs: sleep(100)   |  mem-estimate=0B mem-reservation=0B thread-reservation=0   |   00:UNION      constant-operands=1      mem-estimate=0B mem-reservation=0B thread-reservation=0      tuple-ids=0 row-size=1B cardinality=1      in pipelines: <none>   ----------------       Estimated Per-Host Mem: 10485760       Request Pool: default-pool       Per Host Min Memory Reservation: shared-centos64-ec2-m2-4xlarge-ondemand-0691.vpc.cloudera.com:22000(0)       Per Host Number of Fragment Instances: shared-centos64-ec2-m2-4xlarge-ondemand-0691.vpc.cloudera.com:22000(1)       Admission result: Timed out (queued)       Initial admission queue reason: waited 60000 ms, reason: queue is not empty (size 2); queued queries are executed first. Warning: admission control information from statestore is stale: 1s258ms since last update was received.       Latest admission queue reason: number of running queries 1 is at or over limit 1 (configured statically) Warning: admission control information from statestore is stale: 3s348ms since last update was received..       Query Compilation: 3.468ms          - Metadata of all 0 tables cached: 615.388us (615.388us)          - Analysis finished: 1.299ms (683.787us)          - Authorization finished (noop): 1.377ms (78.537us)          - Value transfer graph computed: 1.723ms (345.718us)          - Single node plan created: 1.825ms (101.771us)          - Distributed plan created: 1.889ms (64.414us)          - Planning finished: 3.468ms (1.579ms)       Query Timeline: 1m          - Query submitted: 0.000ns (0.000ns)          - Planning finished: 4.000ms (4.000ms)          - Submit for admission: 5.000ms (1.000ms)          - Queued: 6.000ms (1.000ms)          - Completed admission: 1m (59s999ms)          - Rows available: 1m (0.000ns)        - ComputeScanRangeAssignmentTimer: 0.000ns       Frontend:     ImpalaServer:        - ClientFetchWaitTimer: 0.000ns        - NumRowsFetched: 0 (0)        - NumRowsFetchedFromCache: 0 (0)        - RowMaterializationRate: 0        - RowMaterializationTimer: 0.000ns    assert 'Admission result: Admitted (queued)' in 'Query (id=a644e4b355da6009:434111e500000000):\n  DEBUG MODE WARNING: Query profile created while running a DEBUG buil...0)\n     - NumRowsFetchedFromCache: 0 (0)\n     - RowMaterializationRate: 0\n     - RowMaterializationTimer: 0.000ns\n'
      

      Attachments

        Activity

          People

            bikramjeet.vig Bikramjeet Vig
            afan Alice Fan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: