[NEMO-479] Approximate the factors that affect the stage group level execution time

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: gsoc2023
• Assignee: Unassigned
• Reporter: Lee Hae Dong
• Created: 12/Apr/21 12:51
• Updated: 09/Mar/23 08:53

Description

There are some factors that can affect the stage group level simulation, such as a latency, the rate of skewed data and the error rate of the executor etc. It is required to find a reasonable distribution form for these factors. Such as the normal distribution or the landau distribution. In actual running, It makes it possible to approximate the model with a small amount of data.

Issue Links

• is required by: NEMO-464 Implement an accurate task execution simulator to predict distributed data processing execution