Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14551 Unaligned checkpoints
  3. FLINK-14498

Introduce NetworkBufferPool#isAvailable() for non-blocking output

    XMLWordPrintableJSON

    Details

      Description

      In order to best-effort implement non-blocking output, we need to further improve the interaction between LocalBufferPool and NetworkBufferPool in non-blocking way as a supplementation of FLINK-14396.

      In detail, we provide the NetworkBufferPool#isAvailable to indicate the global pool state, then we could combine its state via LocalBufferPool#isAvailable` method to avoid blocking in global request while task processing.

      Meanwhile we would refactor the process when LocalBufferPool requests global buffer. If there are no available buffers in NetworkBufferPool, the LocalBufferPool should monitor the global's available future instead of waiting 2 seconds currently in every loop retry. So we can solve the wait delay and cleanup the codes in a unified way.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kevin.cyj Yingjie Cao
                Reporter:
                zjwang Zhijiang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m