Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-28374 Some further improvements of blocking shuffle
  3. FLINK-28519

Fix the bug that SortMergeResultPartitionReadScheduler may not read data sequentially

    XMLWordPrintableJSON

Details

    Description

      Currently, the SortMergeResultPartitionReadScheduler always gets all active subpartition readers and read at most one data region for them. It is common that some subpartitions are requested before others and their region indexes are ahead of others. If all region data of a subpartition can be read in one round, some subpartition readers will always ahead of others which will cause random IO. This patch fixes this case by polling one subpartition reader at a time.

      Attachments

        Issue Links

          Activity

            People

              tanyuxin Yuxin Tan
              kevin.cyj Yingjie Cao
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: