Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-24757

Yarn application is not terminated after the job finishes when submitting a yarn-per-job insert job with SQL client

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.14.0
    • 1.15.0
    • Table SQL / Client
    • None

    Description

      I've seen this problem for about three times in the user mailing thread (previously I suspect that the users are specifying the wrong execution.target) until I myself also bumped into this problem. I've submitted a yarn-per-job batch insert SQL with Flink SQL client and after the job finishes Yarn application is not terminated.

      This is because yarn job cluster is using MiniDispatcher and it will directly terminate only in detached execution mode. This execution mode is (through some function calls) related to DeploymentOptions#ATTACHED which is true by default if jobs are submitted from SQL client.

      When submitting an insert job, SQL client will not wait for the job to finish. Instead it only reports the job id. So I think it is reasonable to set detached mode for every insert job.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            TsReaper Caizhi Weng

            Dates

              Created:
              Updated:

              Issue deployment