Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16069

Creation of TaskDeploymentDescriptor can block main thread for long time

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • None
    • Runtime / Coordination
    • None

    Description

      The deploy of tasks will take long time when we submit a high parallelism job. And Execution#deploy run in mainThread, so it will block JobMaster process other akka messages, such as Heartbeat. The creation of TaskDeploymentDescriptor take most of time. We can put the creation in future.

      For example, A job [source(8000)->sink(8000)], the total 16000 tasks from SCHEDULED to DEPLOYING took more than 1mins. This caused the heartbeat of TaskManager timeout and job never success.

      Attachments

        1. batch.png
          145 kB
          Zhilong Hong
        2. streaming.png
          142 kB
          Zhilong Hong
        3. FLINK-16069-POC-results
          2 kB
          Zhu Zhu

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue FLINK-23005 Cache the compressed serialized value of ShuffleDescriptors

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue FLINK-23005 Cache the compressed serialized value of ShuffleDescriptors

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              huwh !huwh
              Votes:
              2 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: