Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2442

Multiple Stores in pig streaming causes infinite waiting

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1, 0.9.0
    • Fix Version/s: 0.10.0, 0.9.3, 0.11
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Hi,

      If there are multiple store in a pig streaming script, it goes into infinite waiting.

      Script

      DEFINE SCRIPT `./a.pl` SHIP ('/homes/anithar/a.pl');;
      DEFINE SCRIPT1 `./b.pl` SHIP ('/homes/anithar/b.pl');;
      A = LOAD 'test.txt' USING PigStorage() ;
      B1 = STREAM A THROUGH SCRIPT ;
      B1 = foreach B1 generate $0;
      STORE B1 INTO 'B1' USING PigStorage();
      B2 =  STREAM B1 THROUGH SCRIPT1;
      STORE B2 INTO 'B2' USING PigStorage();
      

      a.pl
      --------
      #! /usr/bin/perl -w
      while (my $line = <STDIN>) {
      print uc($line);
      }
      --------

      b.pl
      ---------
      #! /usr/bin/perl -w
      while (my $line = <STDIN>)

      { print $line; } --------- Input (test.txt) {code}

      test
      hi
      hello

      
      

      This infinite waiting happens randomly causing the job to fail with "Task attempt failed to report
      status for 605 seconds. Killing!".
      Same happens with 0.8 version too.

      Regards,
      Anitha

        Attachments

        1. PIG-2442-1.patch
          6 kB
          Daniel Dai
        2. PIG-2442_pig9.patch
          5 kB
          Jonathan Coveney

          Activity

            People

            • Assignee:
              xutingz xuting zhao
              Reporter:
              anitharaju Anitha Raju
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: