Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8581

Improve performance for low latency network

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.0
    • Fix Version/s: None
    • Component/s: Network
    • Labels:
      None

      Attachments

        Issue Links

        1.
        Introduce BufferConsumer concept Sub-task Closed Piotr Nowojski
        2.
        Pass BufferConsumer to subpartitions Sub-task Closed Piotr Nowojski
        3.
        Handle read-only buffers in DataInputDeserializer Sub-task Closed Piotr Nowojski
        4.
        Drop SpilledSubpartitionViewTest Sub-task Closed Piotr Nowojski
        5.
        Drop unused AdaptiveSpanningRecordDeserializer Sub-task Closed Piotr Nowojski
        6.
        Handle sliced buffers in RecordDeserializer Sub-task Closed Piotr Nowojski
        7.
        Add data polling method to InputGate Sub-task Closed Piotr Nowojski
        8.
        Drop ResultPartitionWriter#writeBufferToAllSubpartitions method Sub-task Closed Piotr Nowojski
        9.
        Handle unfinished BufferConsumers in subpartitions Sub-task Closed Piotr Nowojski
        10.
        Move OutputFlusher thread to Netty scheduled executor Sub-task In Progress Piotr Nowojski
        11.
        Efficiently handle alwaysFlush case (0ms flushTimeout) Sub-task Closed Piotr Nowojski
        12.
        Make notifyDataAvailable call reliable Sub-task Closed Piotr Nowojski
        13.
        SpillableSubpartition#spillFinishedBufferConsumers() does not count spilled bytes anymore Sub-task Resolved Nico Kruber
        14.
        SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView Sub-task Resolved Nico Kruber
        15.
        Memory segment offsets for slices of slices are wrong Sub-task Resolved Nico Kruber
        16.
        Creating a union of UnionGate instances will fail with UnsupportedOperationException when retrieving buffers Sub-task Resolved Nico Kruber
        17.
        InputGate may contain data after an EndOfPartitionEvent Sub-task Closed Piotr Nowojski
        18.
        SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available Sub-task Closed Nico Kruber
        19.
        Correctly set `moreAvailable` flag in SingleInputGate and UnionInputGate Sub-task Closed Piotr Nowojski
        20.
        SpillableSubpartitionView#getNextBuffer always sets isMoreAvailable to false when switching from spillable to spilled Sub-task Closed Nico Kruber
        21.
        Add network profiling/diagnosing metrics. Sub-task Open Unassigned

          Activity

            People

            • Assignee:
              pnowojski Piotr Nowojski
              Reporter:
              pnowojski Piotr Nowojski
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: