Description
This issue was discovered during https://github.com/apache/spark/pull/21738 .
It turns out that limit is not whole-stage-codegened correctly and always consume all the inputs
Attachments
Issue Links
- is duplicated by
-
SPARK-25597 SQL query with limit iterates the whole iterator when WholeStage code generation is enabled
- Resolved
-
SPARK-26280 Spark will read entire CSV file even when limit is used
- Resolved
- links to