Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5486 Port control-plane parts of ImpalaInternalService to KRPC
  3. IMPALA-6984

Coordinator should cancel backends when returning EOS

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • Impala 3.0
    • None
    • Backend

    Description

      Currently, the Coordinator waits for backends rather than proactively cancelling them in the case of hitting EOS. There's a tangled mess that makes it tricky to proactively cancel the backends related to how Coordinator::ComputeQuerySummary() works – we can't update the summary until the profiles are no longer changing (which also makes sense given that we want the exec summary to be consistent with the final profile).  But we current tie together the FIS status and the profile, and cancellation of backends causes the FIS to return CANCELLED, which then means that the remaining FIS on that backend won't produce a final profile.

      With the rework of the protocol for IMPALA-2990 we should make it possible to sort this out such that a final profile can be requested regardless of how a FIS ends execution.

      This also relates to IMPALA-5783.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-9755 Flaky test: test_global_exchange_counters

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          depends upon

          Sub-task - The sub-task of the issue IMPALA-5119 Don't make RPCs from Coordinator::UpdateBackendExecStatus()

          • Major - Major loss of function.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-2990 Coordinator should timeout and cancel queries with unresponsive / stuck executors

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-5783 Investigate exec summary behavior

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-539 Impala should gather final runtime profile from fragments for aborted/cancelled query

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue IMPALA-5119 Don't make RPCs from Coordinator::UpdateBackendExecStatus()

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue IMPALA-8845 Close ExecNode tree prior to calling FlushFinal in FragmentInstanceState

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-9113 Queries can hang if an impalad is killed after a query has FINISHED

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. IMPALA-9124 Transparently retry queries that fail due to cluster membership changes

          • Critical - Crashes, loss of data, severe memory leak.
          • In Progress

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-3990 ExchangeNode::Close() cancel sender fragment if called before eos

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              wzhou Wenzhe Zhou
              dhecht Daniel Hecht
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: