Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3760 Predicate pushdown for columnar file formats
  3. PIG-4096

Implement predicate pushdown evaluation in OrcStorage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 0.18.0
    • None
    • None

    Description

      Parquest supports predicate evaluation and filtering of records natively via https://github.com/apache/incubator-parquet-mr/blob/master/parquet-column/src/main/java/parquet/filter2/predicate/FilterApi.java. It would be good to have it in ORC Reader as well but is not available right now. So evaluate the predicate pushdown condition in OrcStorage itself and filter records. This requires PIG-4093.

      Attachments

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue PIG-4093 Predicate pushdown to support removing filters from pig plan

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              rohini Rohini Palaniswamy
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: