Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16428

Fine-grained network buffer management for backpressure

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.14.0
    • Component/s: Runtime / Network
    • Labels:
      None

      Description

      It is an umbrella ticket for tracing the progress of this improvement.

      This is the second ingredient to solve the “checkpoints under backpressure” problem (together with unaligned checkpoints). It consists of two steps:

      • See if we can use less network memory in general for streaming jobs (with potentially different distribution of floating buffers in the input side)
      • Under backpressure, reduce network memory to have less in-flight data (needs specification of algorithm and experiments)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zjwang Zhijiang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h