[NEMO-443] Efficient Caching and Spilling on Nemo

• Type: Umbrella
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: full-time gsoc gsoc2022 gsoc2023
• Assignee: Unassigned
• Reporter: Jeongyoon Eo
• Created: 23/Mar/20 04:53
• Updated: 09/Mar/23 08:52

Description

In-memory caching and spilling are essential features in in-memory big data processing frameworks, and Nemo needs one.

• Identify and persist frequently used data and unpersist it when its usage ended
• Spill in-memory data to disk upon memory pressure

Issue Links

• incorporates: NEMO-444 Implement spill mechanism on Nemo
• incorporates: NEMO-445 Application structure-aware caching on Nemo