Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-10363

test_mixed_catalog_ddls_with_invalidate_metadata failed after reaching timeout (120 seconds)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Catalog
    • Epic Color:
      ghx-label-3

      Description

      We found in an s3 run that the E2E test test_mixed_catalog_ddls_with_invalidate_metadata at https://github.com/apache/impala/blob/6539c50f6611d9e2ba9c3051b8e4c8bd6e79d1e1/tests/custom_cluster/test_concurrent_ddls.py#L58-L59 could fail after reaching timeout of 120 seconds with the following error message.

      Error Message
      Failed: Timeout >120.0s
      Stacktrace
      custom_cluster/test_concurrent_ddls.py:59: in test_mixed_catalog_ddls_with_invalidate_metadata
          self._run_ddls_with_invalidation(unique_database, sync_ddl=False)
      custom_cluster/test_concurrent_ddls.py:146: in _run_ddls_with_invalidation
          for i in pool.imap_unordered(run_ddls, xrange(1, NUM_ITERS + 1)):
      /usr/lib64/python2.7/multiprocessing/pool.py:642: in next
          self._cond.wait(timeout)
      /usr/lib64/python2.7/threading.py:339: in wait
          waiter.acquire()
      E   Failed: Timeout >120.0s
      
      Error Message
      Failed: Timeout >120.0s
      Stacktrace
      custom_cluster/test_concurrent_ddls.py:59: in test_mixed_catalog_ddls_with_invalidate_metadata
          self._run_ddls_with_invalidation(unique_database, sync_ddl=False)
      custom_cluster/test_concurrent_ddls.py:146: in _run_ddls_with_invalidation
          for i in pool.imap_unordered(run_ddls, xrange(1, NUM_ITERS + 1)):
      /usr/lib64/python2.7/multiprocessing/pool.py:642: in next
          self._cond.wait(timeout)
      /usr/lib64/python2.7/threading.py:339: in wait
          waiter.acquire()
      E   Failed: Timeout >120.0s
      

      Based on the following two lines in the console output of the Jenkins job, it looks like the query id of the failed query is

      08:20:05 custom_cluster/test_concurrent_ddls.py::TestConcurrentDdls::test_mixed_catalog_ddls_with_invalidate_metadata -- 2020-11-27 08:20:05,777 INFO     Thread-85: Started query 4b4d32c23dedf060:21e898dd00000000
      08:21:14 FAILED
      

      It looks like there were multiple queries being executed during this test. In this regard, I provide the corresponding log files generated when the test was executed. Let me know if other log files are needed as well. Thanks!

      This test was recently added in IMPALA-7506. Maybe Quanlong Huang could provide some insight into it.

      This JIRA may also be related to IMPALA-10301.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stigahuang Quanlong Huang
                Reporter:
                fangyurao Fang-Yu Rao
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: