Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3510

New filter extractor fails with more than one filter statement

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.12.1
    • Component/s: impl
    • Labels:
      None

      Description

      This is a regression from PIG-3461 - rewrite of partition filter optimizer. Here is an example that demonstrates the problem:

      two filters
      b = FILTER a BY (dateint >= 20130901 AND dateint <= 20131001);
      c = FILTER b BY (event_id == 419 OR event_id == 418);
      
      one filter
      b = FILTER a BY (dateint >= 20130901 AND dateint <= 20131001) AND (event_id == 419 OR event_id == 418);
      

      Both dateint and event_id are partition columns. For the 1 filter case, the whole expression is pushed down whereas for the 2 filter case, only (event_id == 419 OR event_id == 418) is pushed down.

        Attachments

        1. PIG-3510.patch
          2 kB
          Aniket Namadeo Mokashi
        2. PIG-3510-2.patch
          4 kB
          Cheolsoo Park

          Issue Links

            Activity

              People

              • Assignee:
                aniket486 Aniket Namadeo Mokashi
                Reporter:
                cheolsoo Cheolsoo Park
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: