Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-401

SplitIntroducer does not handle some cases correctly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.2.0
    • 0.2.0
    • None
    • None

    Description

      The splitIntroducer does not handle the following cases correctly:

      • As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops - it should still recursively check the successors:
        Current code which needs to change:
                List<LogicalOperator> sucs = mPlan.getSuccessors(root);
                if(sucs==null) return; 
                int size = sucs.size();
                if(size==0 || size==1) return;  --> THIS should change to recursively process the succesor if it exists(size == 1 case)
        
      • If there are more than one operators in the plan which need implicit splits to be introduced
      • If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for it's inputs and GroupBy expressions needs to be updated
      • The kind of changes need for cogroup maybe needed for some other operators

      I started looking at it some and have a patch for the first two issues - UNTESTED which I am attaching to this issue

      Attachments

        1. PIG-401-partial.patch
          2 kB
          Pradeep Kamath
        2. PIG401.patch
          15 kB
          Alan Gates

        Activity

          People

            gates Alan Gates
            pkamath Pradeep Kamath
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: