Details

      Description

      For in-predicates with long in-lists such as TPCDS-Q8, we may benefit by finding the min and max in the list and use them to prune out unqualified scan ranges or filter out rows which don't fall into the range.

        Issue Links

          Activity

          Hide
          lv Lars Volker added a comment -

          Tests for this should also make sure, that the equivalent min/max predicates make use of the Parquet statistics.

          Show
          lv Lars Volker added a comment - Tests for this should also make sure, that the equivalent min/max predicates make use of the Parquet statistics.
          Hide
          joemcdonnell Joe McDonnell added a comment -

          commit aa05c6493b0ff8bbf422a4c38cf780bde34d51c7
          Author: Joe McDonnell <joemcdonnell@cloudera.com>
          Date: Fri Apr 14 11:59:08 2017 -0700

          IMPALA-3654: Parquet stats filtering for IN predicate

          This generates min/max predicates for InPredicates that
          have only constant values in the IN list. It is only
          used for statistics filtering on Parquet files.

          Change-Id: I4a88963a7206f40a867e49eceeaf03fdd4f71997
          Reviewed-on: http://gerrit.cloudera.org:8080/6810
          Reviewed-by: Alex Behm <alex.behm@cloudera.com>
          Tested-by: Impala Public Jenkins

          Show
          joemcdonnell Joe McDonnell added a comment - commit aa05c6493b0ff8bbf422a4c38cf780bde34d51c7 Author: Joe McDonnell <joemcdonnell@cloudera.com> Date: Fri Apr 14 11:59:08 2017 -0700 IMPALA-3654 : Parquet stats filtering for IN predicate This generates min/max predicates for InPredicates that have only constant values in the IN list. It is only used for statistics filtering on Parquet files. Change-Id: I4a88963a7206f40a867e49eceeaf03fdd4f71997 Reviewed-on: http://gerrit.cloudera.org:8080/6810 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Impala Public Jenkins
          Hide
          lv Lars Volker added a comment -

          Are there plans to extend this to other file formats than Parquet?

          Show
          lv Lars Volker added a comment - Are there plans to extend this to other file formats than Parquet?

            People

            • Assignee:
              joemcdonnell Joe McDonnell
              Reporter:
              kwho Michael Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development