[SPARK-26975] Support nested-column pruning over limit/sample/repartition

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 22/Feb/19 21:30
• Updated: 24/Feb/20 21:45
• Resolved: 20/Mar/19 03:25

Description

As SPARK-26958 shows the benchmark, nested-column pruning has limitations. This issue aims to remove the limitations on `limit/repartition/sample`. In this issue, repartition means `Repartition`, not `RepartitionByExpression`.

Issue Links

• causes: SPARK-30870 catalyst inception of lateral view explode with struct raise a Catalyst error
• links to: GitHub Pull Request #23873
• links to: GitHub Pull Request #23964
• links to: GitHub Pull Request #26975