Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3173

Partition filter push down does not happen partition keys condition include a AND and OR construct

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      A = load 'db.table' using org.apache.hcatalog.pig.HCatLoader();
      B = filter A by (region=='usa' AND dt=='201302051800') OR (region=='uk' AND dt=='201302051800');
      C = foreach B generate name, age;
      DUMP C;

      gives the below warning and scans the whole table.

      2013-02-06 22:22:16,233 [main] WARN org.apache.pig.newplan.PColFilterExtractor - No partition filter push down: You have an partition column (region ) in a construction like: (pcond and ...) or (pcond and ...) where pcond is a condition on a partition column.
      2013-02-06 22:22:16,233 [main] WARN org.apache.pig.newplan.PColFilterExtractor - No partition filter push down: You have an partition column (datestamp ) in a construction like: (pcond and ...) or (pcond and ...) where pcond is a condition on a partition column.

        Attachments

        1. PIG-3173-1.patch
          13 kB
          Rohini Palaniswamy
        2. PIG-3173-2.patch
          9 kB
          Rohini Palaniswamy

          Issue Links

            Activity

              People

              • Assignee:
                rohini Rohini Palaniswamy
                Reporter:
                rohini Rohini Palaniswamy
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: