Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8581

Improve performance for low latency network

    XMLWordPrintableJSON

Details

    Attachments

      Issue Links

        supercedes

        Improvement - An improvement or enhancement to an existing feature or task. FLINK-7612 Implement Event-driven Network IO

        • Major - Major loss of function.
        • Closed
        links to

        Web Link GitHub Pull Request #5423

        Web Link GitHub Pull Request #6698

        1.
        		 Introduce BufferConsumer concept Sub-task Closed Piotr Nowojski
        2.
        		 Pass BufferConsumer to subpartitions Sub-task Closed Piotr Nowojski
        3.
        		 Handle read-only buffers in DataInputDeserializer Sub-task Closed Piotr Nowojski
        4.
        		 Drop SpilledSubpartitionViewTest Sub-task Closed Piotr Nowojski
        5.
        		 Drop unused AdaptiveSpanningRecordDeserializer Sub-task Closed Piotr Nowojski
        6.
        		 Handle sliced buffers in RecordDeserializer Sub-task Closed Piotr Nowojski
        7.
        		 Add data polling method to InputGate Sub-task Closed Piotr Nowojski
        8.
        		 Drop ResultPartitionWriter#writeBufferToAllSubpartitions method Sub-task Closed Piotr Nowojski
        9.
        		 Handle unfinished BufferConsumers in subpartitions Sub-task Closed Piotr Nowojski
        10.
        		 Efficiently handle alwaysFlush case (0ms flushTimeout) Sub-task Closed Piotr Nowojski
        11.
        		 Make notifyDataAvailable call reliable Sub-task Closed Piotr Nowojski
        12.
        		 SpillableSubpartition#spillFinishedBufferConsumers() does not count spilled bytes anymore Sub-task Resolved Nico Kruber
        13.
        		 SpillableSubpartition does not update byte statistics when spilled by a SpillableSubpartitionView Sub-task Resolved Nico Kruber
        14.
        		 Memory segment offsets for slices of slices are wrong Sub-task Resolved Nico Kruber
        15.
        		 Creating a union of UnionGate instances will fail with UnsupportedOperationException when retrieving buffers Sub-task Resolved Nico Kruber
        16.
        		 InputGate may contain data after an EndOfPartitionEvent Sub-task Closed Piotr Nowojski
        17.
        		 SpilledSubpartitionView wrongly relys on the backlog for determining whether more data is available Sub-task Closed Nico Kruber
        18.
        		 Correctly set `moreAvailable` flag in SingleInputGate and UnionInputGate Sub-task Closed Piotr Nowojski
        19.
        		 SpillableSubpartitionView#getNextBuffer always sets isMoreAvailable to false when switching from spillable to spilled Sub-task Closed Nico Kruber
        20.
        		 Add network profiling/diagnosing metrics. Sub-task Closed boshu Zheng
        21.
        		 Reduce number of flush requests to the network stack Sub-task Closed Nico Kruber
        22.
        		 Move data available notification in PipelinedSubpartition out of the synchronized block Sub-task Closed Nico Kruber
        23.
        		 Notify data available to network stack immediately after finishing BufferBuilder Sub-task Closed Zhijiang

        Activity

          People

            pnowojski Piotr Nowojski
            pnowojski Piotr Nowojski
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: