Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4463

Hang in test_mini_stress.py with MT_DOP=3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: None
    • Component/s: Backend
    • Labels:

      Description

      A Jenkins run hung while running test_mini_stress.py::test_mini_stress.

      Observations at the time of the hang:

      • There are a total of 64 FE connections in use.
      • There are 15 active queries that returned results. Their latest timeline event is "UnregisterQuery" from which there is no more progress.

      Attached:

      • Backtraces of all threads of the impalad processes at the time of the hang
      • Raw JSON of the impalad metrics at the time of the hang

      I have not yet tried to reproduce locally. This is my first run of this test in this configuration, so I assume it should be easy to reproduce.

      bin/start-impala-cluster.py --impalad_args="--default_query_options=mt_dop=3"
      tests/run-tests.py stress/test_mini_stress.py
      

        Attachments

        1. impalad0_metrics.txt
          69 kB
          Alexander Behm
        2. impalad2_metrics.txt
          55 kB
          Alexander Behm
        3. impalad1_metrics.txt
          55 kB
          Alexander Behm
        4. 2292_stacks.txt
          1.40 MB
          Alexander Behm
        5. 2321_stacks.txt
          1.28 MB
          Alexander Behm
        6. 2354_stacks.txt
          1.28 MB
          Alexander Behm

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: