[CARBONDATA-3994] Skip Order by for map task if it is sort column and use limit pushdown for array_contains filter

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Assignee: Ajantha Bhat
• Reporter: Ajantha Bhat
• Created: 16/Sep/20 14:25
• Updated: 23/Oct/20 05:29
• Resolved: 23/Oct/20 05:29

Description

When the order by column is in sort column, every map task output will be already sorted. No need to sort the data again.
Hence skipping the order at map task by changing plan node from TakeOrderedAndProject --> CarbonTakeOrderedAndProjectExec
Also in this scenario collecting the limit at map task and Array_contains() will use this limit value for row scan filtering to break scan once limit value is reached.
Also added a carbon property to control this .
carbon.mapOrderPushDown.<db_name>_<table_name>.column

Note: later we can improve this for other filters also to use the limit value.