Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5118

Script fails with Invalid dag containing 0 vertices

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0, 0.16.1
    • Component/s: tez
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The current code of splitting DAG does not handle cases where there is split and a clean segmentation is not possible. It also does not handle cases where there is multiple segments and they are later used together. Both cases it ends up with "Invalid dag containing 0 vertices" error as it creates one empty DAG and and a new DAG which contains the full original plan as it was not able to segment.

      Usually can ask user to get rid of unnecessary store and then load if in same script as it is usually a result of bad programming and inefficient. But in this case they were part of different large scripts and run as below.

      script.pig
      run -param output=output1 script1.pig  --stores to output1
      run -param input=output1 script2.pig  -- load from output1
      

        Attachments

        1. PIG-5118.fixTest.patch
          3 kB
          Ádám Szita
        2. PIG-5118-1.patch
          31 kB
          Rohini Palaniswamy
        3. PIG-5118-2.patch
          39 kB
          Rohini Palaniswamy
        4. PIG-5118-3.patch
          43 kB
          Rohini Palaniswamy

          Issue Links

            Activity

              People

              • Assignee:
                rohini Rohini Palaniswamy
                Reporter:
                rohini Rohini Palaniswamy
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: