Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14799 Improve batch sql and hive integrate performance milestone-2
  3. FLINK-14860

Provide a new InputSplitAssigner that consider page cache utilization ratio

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Connectors / FileSystem
    • None

    Description

      At present, only the need of localization is considered in LocatableInputSplitAssigner, but there is a problem of random floating among three copies of split (HDFS has three copies for files), which leads to excessive use of cache. Therefore, we also need to make it possible to schedule three copies in a clear order on the basis of localization.

      Attachments

        Activity

          People

            Unassigned Unassigned
            lzljs3620320 Jingsong Lee
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: