[NEMO-477] Implement a model that represent a task level exeuction time with statistical analysis

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: gsoc2023
• Assignee: Unassigned
• Reporter: Lee Hae Dong
• Created: 12/Apr/21 12:37
• Updated: 09/Mar/23 08:53

Description

The current SimulatedTaskExecutor is hardly available. because it needs actual metric to predict execution time. To increase utilization, we need new model that predicts a task level execution time with statistical analysis. 

Some of the related TODOs are as follows:

• Find factors that affect a task level execution time. with loose grid search.
• Infer the most suitable model with tight grid search. 

Issue Links

• is required by: NEMO-464 Implement an accurate task execution simulator to predict distributed data processing execution