[SPARK-22348] The table cache providing ColumnarBatch should also do partition batch pruning

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 25/Oct/17 02:08
• Updated: 25/Oct/17 05:34
• Resolved: 25/Oct/17 05:34

Description

We enable table cache InMemoryTableScanExec to provide ColumnarBatch now. But the cached batches are retrieved without pruning. In this case, we still need to do partition batch pruning.

Issue Links

• links to: [Github] Pull Request #19569 (viirya)