Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-2834

Global round-robin for temporary directories

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.9.2, 0.10.0
    • Component/s: Runtime / Coordination
    • Labels:
      None

      Description

      Currently each FileIOChannel performs a local round-robin among temporary directories. Initially every writer will be concurrently spilling to the same first directory, then the same second directory, etc. Even when evenly dispersed this effect will naturally occur due to the bunching effect.

      One solution is to use a global counter when selecting the temporary path.

        Attachments

          Activity

            People

            • Assignee:
              greghogan Greg Hogan
              Reporter:
              greghogan Greg Hogan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: