[SPARK-45563] Spark history files backend currently depend on polling for loading into the history server

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: UI
• Target Version/s: 4.0.0
• Assignee: Holden Karau
• Reporter: Holden Karau
• Created: 16/Oct/23 23:03
• Updated: 16/Oct/23 23:10

Description

The spark history server FS  currently depends on polling for loading history files but we should support on demand loading as well.