Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22348

The table cache providing ColumnarBatch should also do partition batch pruning

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None

      Description

      We enable table cache InMemoryTableScanExec to provide ColumnarBatch now. But the cached batches are retrieved without pruning. In this case, we still need to do partition batch pruning.

        Attachments

          Activity

            People

            • Assignee:
              viirya Liang-Chi Hsieh
              Reporter:
              viirya Liang-Chi Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: