Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1178

LogicalPlan and Optimizer are too complex and hard to work with


    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:


      The current implementation of the logical plan and the logical optimizer in Pig has proven to not be easily extensible. Developer feedback has indicated that adding new rules to the optimizer is quite burdensome. In addition, the logical plan has been an area of numerous bugs, many of which have been difficult to fix. Developers also feel that the logical plan is difficult to understand and maintain. The root cause for these issues is that a number of design decisions that were made as part of the 0.2 rewrite of the front end have now proven to be sub-optimal. The heart of this proposal is to revisit a number of those proposals and rebuild the logical plan with a simpler design that will make it much easier to maintain the logical plan as well as extend the logical optimizer.

      See http://wiki.apache.org/pig/PigLogicalPlanOptimizerRewrite for full details.


        1. expressions.patch
          36 kB
          Alan Gates
        2. expressions-2.patch
          37 kB
          Alan Gates
        3. lp.patch
          84 kB
          Ying He
        4. lp.patch
          108 kB
          Ying He
        5. pig_1178_2.patch
          180 kB
          Ankit Modi
        6. pig_1178_3.2.patch
          342 kB
          Daniel Dai
        7. pig_1178_3.3.patch
          342 kB
          Daniel Dai
        8. pig_1178_3.4.patch
          86 kB
          Daniel Dai
        9. pig_1178_3.patch
          342 kB
          Ankit Modi
        10. pig_1178.patch
          435 kB
          Ankit Modi
        11. pig_1178.patch
          304 kB
          Ankit Modi
        12. PIG_1178.patch
          110 kB
          Ying He
        13. PIG-1178-10.patch
          23 kB
          Daniel Dai
        14. PIG-1178-11.patch
          44 kB
          Daniel Dai
        15. PIG-1178-4.patch
          522 kB
          Daniel Dai
        16. PIG-1178-5.patch
          1.70 MB
          Daniel Dai
        17. PIG-1178-6.patch
          287 kB
          Daniel Dai
        18. PIG-1178-7.patch
          239 kB
          Daniel Dai
        19. PIG-1178-8.patch
          8 kB
          Daniel Dai
        20. PIG-1178-9.patch
          2 kB
          Daniel Dai

          Issue Links



              • Assignee:
                daijy Daniel Dai
                alangates Alan Gates
              • Votes:
                0 Vote for this issue
                2 Start watching this issue


                • Created: