Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-24757

Yarn application is not terminated after the job finishes when submitting a yarn-per-job insert job with SQL client

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.14.0
    • 1.15.0
    • Table SQL / Client
    • None


      I've seen this problem for about three times in the user mailing thread (previously I suspect that the users are specifying the wrong execution.target) until I myself also bumped into this problem. I've submitted a yarn-per-job batch insert SQL with Flink SQL client and after the job finishes Yarn application is not terminated.

      This is because yarn job cluster is using MiniDispatcher and it will directly terminate only in detached execution mode. This execution mode is (through some function calls) related to DeploymentOptions#ATTACHED which is true by default if jobs are submitted from SQL client.

      When submitting an insert job, SQL client will not wait for the job to finish. Instead it only reports the job id. So I think it is reasonable to set detached mode for every insert job.



          This comment will be Viewable by All Users Viewable by All Users


            Unassigned Unassigned
            TsReaper Caizhi Weng



              Issue deployment