[CARBONDATA-4195] Materialized view loading time increased due to full refresh

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.0
• Component/s: core
• Assignee: Unassigned
• Reporter: Mayuri Patole
• Created: 31/May/21 09:18
• Updated: 24/Aug/21 05:14

Description

Hi Team,

We are using carbon 2.1.0 in our project where parallel data loading is happening.

We are working on getting optimal performance for aggregated queries using materialized views.

We observed that continues data loading and full refresh of MV is causing increased load time and high memory usage which doesn't have to be this way.

Can you suggest a way to perform incremental refresh because we do not need to calculate old data again while loading ?