[SPARK-34509] Make dynamic allocation upscaling more progressive on K8S

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.4, 2.4.7, 3.0.2, 3.2.0
• Component/s: Kubernetes
• Assignee: Unassigned
• Reporter: Attila Zsolt Piros
• Created: 23/Feb/21 16:09
• Updated: 09/Mar/21 20:41

Description

Currently even a single late pod request stops upscaling. As we have allocation batch size it would be better to go up to that limit as soon as possible (if serving pod requests are slow we have to make them as early as possible).

Issue Links

• links to: [Github] Pull Request #31790 (attilapiros)