[FLINK-15467] Should wait for the end of the source thread during the Task cancellation - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.9.0, 1.9.1, 1.10.1
Fix Version/s: 1.10.3, 1.11.2, 1.12.0
Component/s: Runtime / Task
Labels:
- pull-request-available

Description

In the new mailBox model, SourceStreamTask starts a source thread to run user methods, and the current execution thread will block on mailbox.takeMail (). When a task cancels, the TaskCanceler thread will cancel the task and interrupt the execution thread. Therefore, the execution thread of SourceStreamTask will throw InterruptedException, then cancel the task again, and throw an exception.

//代码占位符
@Override
protected void performDefaultAction(ActionContext context) throws Exception {
   // Against the usual contract of this method, this implementation is not step-wise but blocking instead for
   // compatibility reasons with the current source interface (source functions run as a loop, not in steps).
   sourceThread.start();

   // We run an alternative mailbox loop that does not involve default actions and synchronizes around actions.
   try {
      runAlternativeMailboxLoop();
   } catch (Exception mailboxEx) {
      // We cancel the source function if some runtime exception escaped the mailbox.
      if (!isCanceled()) {
         cancelTask();
      }
      throw mailboxEx;
   }

   sourceThread.join();
   if (!isFinished) {
      sourceThread.checkThrowSourceExecutionException();
   }

   context.allActionsCompleted();
}

When all tasks of this TaskExecutor are canceled, the blob file will be cleaned up. But the real source thread is not finished at this time, which will cause a ClassNotFoundException when loading a new class. In this case, the source thread may not be able to properly clean up and release resources (such as closing child threads, cleaning up local files, etc.). Therefore, I think we should mark this task canceled or finished after the execution of the source thread is completed.