Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-500

PPD: StripeStatistics::getColumnStatistics() does not column prune

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.5.5
    • Fix Version/s: None
    • Component/s: Java
    • Labels:
      None

      Description

      StripeStatistics::getColumnStatistics() will deserialize all column statistics, even if the ORC predicate push-down and the eventual SARG applies to a subset.

      This is affecting DATE push-down into a very wide table, where the deserialization of the statistics is the host spot during split-generation, due to the new Text() allocation within String statistics implementation.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gopalv Gopal V
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: