Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-441

Dynamic Task Sizing on Nemo

    XMLWordPrintableJSON

    Details

    • Type: Umbrella
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None

      Description

      This is an umbrella issue to keep track of the issues related to the dynamic task sizing feature on Nemo.

      Dynamic task sizing needs to consider a workload and try to decide on the optimal task size based on the runtime metrics and characteristics. It should have an effect on the parallelism and the partitions, on how many partitions an intermediate data should be divided/shuffled into, and to effectively handle skews in the meanwhile.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hwarim Hwarim Hyun
                Reporter:
                wonook Wonook
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: