[IMPALA-7561] Expired queries should end up in a consistent state (EXCEPTION, CANCELLED, or something else)

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: query-lifecycle
• Target Version: Product Backlog
• Epic Color: ghx-label-2
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 11/Sep/18 20:53
• Updated: 25/Feb/21 10:19

Description

ClientRequestState treats "eos" as a terminal state and won't transition to EXCEPTION or CANCELLED out of it. See the below code snippet from ClientRequestState::Cancel():

    bool already_done = eos_ || operation_state_ == TOperationState::ERROR_STATE;
    if (!already_done && cause != NULL) {
      DCHECK(!cause->ok());
      discard_result(UpdateQueryStatus(*cause));
      query_events_->MarkEvent("Cancelled");
      DCHECK_EQ(operation_state_, TOperationState::ERROR_STATE);
    }

If the cancellation is initiated by the server (e.g. because of query_timeout_s or exec_time_limit_s) then we should arguably treat that as an exception and put the query into the EXCEPTION state (because it is not a user-initiated cancellation) if it is not already in an EXCEPTION or CANCELLED state. That way the client can see that the expiry happened and access the cause of expiry.

One argument against using the EXCEPTION state is that admins or users might think that something went wrong and needs to be investigated, see IMPALA-471.

A separate issue is that user-initiated cancellation should use the CANCELLED state: IMPALA-1262

Issue Links

• is related to: IMPALA-8411 A race during query cancellation by a client can make the client see it as still running
• is related to: IMPALA-4714 Idle session expired query goes in to exception state - And this is confusing
• relates to: IMPALA-1262 user initiated query cancellation should return operation_state = CANCELED_STATE