Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3975

Multiple Scalar reference calls leading to missing records

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.8.1, 0.9.2, 0.10.1, 0.11.1, 0.12.2
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      We noticed that multiple pig runs with same input were producing different outputs.

      Simplified script looked like this.

      A = load 'input1' as (a1:int);
      B = group A by a1 parallel 200;
      C = load 'input2' as (c1:int);
      D = foreach C generate B.$0;
      store D into '/tmp/deletemeD';
      E = load 'input3' as (c1:int);
      F = foreach E generate B.$0;
      store F into '/tmp/deletemeF';
      

        Attachments

        1. pig-3975-v03-trunk.patch
          10 kB
          Rohini Palaniswamy
        2. pig-3975-v03-additional-fix.patch
          2 kB
          Rohini Palaniswamy
        3. pig-3975-v03.patch
          10 kB
          Rohini Palaniswamy
        4. pig-3975-v02_withtests.patch
          10 kB
          Koji Noguchi
        5. pig-3975-v01_withouttest.patch
          5 kB
          Koji Noguchi

          Activity

            People

            • Assignee:
              knoguchi Koji Noguchi
              Reporter:
              knoguchi Koji Noguchi
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: