[AURORA-165] Consider using sqlite as a backing store for thermos checkpoints

• Type: Task
• Status: Open
• Priority: Trivial
• Resolution: Unresolved
• Component/s: Thermos
• Assignee: Unassigned
• Reporter: Brian Wickman
• Created: 03/Feb/14 23:21
• Updated: 29/Apr/15 15:17

Description

If a machine gets bogged down with 30,000 dead tasks, replaying the checkpoints can be very costly because they're spread across 30,000 x potentially many different checkpoint files. Investigate to see if there's an abstraction (e.g. a SQLiteRecordWriter) that fits nicely in the system that could provide better performance.