Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17012

Expose stage of task initialization

    XMLWordPrintableJSON

Details

    • Task's RUNNING state was split into two states: INITIALIZING and RUNNING. Task is INITIALIZING while state is initialising and in case of unaligned checkpoints, until all of the in-flight data has been recovered.

    Description

      Currently a task switches to running before fully initialized, does not take state initialization and operator initialization(#open ) in to account, which may take long time to finish. As a result, there would be a weird phenomenon that all tasks are running but throughput is 0.

      I think it could be good if we can expose the initialization stage of tasks. What to you think?

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. FLINK-4815 Automatic fallback to earlier checkpoints when checkpoint restore fails

          • Not a Priority -
          • Open
          causes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-22535 Resource leak would happen if exception thrown during AbstractInvokable#restore of task life

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. FLINK-23034 NPE in JobDetailsDeserializer during the reading old version of ExecutionState

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-22215 Rename RECOVERING state to INITIALIZING

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-20087 CheckpointCoordinator waits until all tasks finish initialization of states to trigger checkpoint

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-20912 Increase Log and Metric: Time consumed by Checkpoint Restore

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-18983 Job doesn't changed to failed if close function has blocked

          • Not a Priority -
          • Reopened

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-22379 Do not trigger checkpoint when non source tasks are INITIALIZING

          • Major - Major loss of function.
          • Resolved
          supercedes

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-4714 Set task state to RUNNING after state has been restored

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #15221

          Web Link GitHub Pull Request #15375

          Web Link GitHub Pull Request #15459

          Web Link GitHub Pull Request #15462

          Activity

            People

              akalashnikov Anton Kalashnikov
              wenlong.lwl Wenlong Lyu
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: