Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2442

Multiple Stores in pig streaming causes infinite waiting

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.8.1, 0.9.0
    • 0.10.0, 0.9.3, 0.11
    • None
    • None
    • Reviewed

    Description

      Hi,

      If there are multiple store in a pig streaming script, it goes into infinite waiting.

      Script

      DEFINE SCRIPT `./a.pl` SHIP ('/homes/anithar/a.pl');;
      DEFINE SCRIPT1 `./b.pl` SHIP ('/homes/anithar/b.pl');;
      A = LOAD 'test.txt' USING PigStorage() ;
      B1 = STREAM A THROUGH SCRIPT ;
      B1 = foreach B1 generate $0;
      STORE B1 INTO 'B1' USING PigStorage();
      B2 =  STREAM B1 THROUGH SCRIPT1;
      STORE B2 INTO 'B2' USING PigStorage();
      

      a.pl
      --------
      #! /usr/bin/perl -w
      while (my $line = <STDIN>) {
      print uc($line);
      }
      --------

      b.pl
      ---------
      #! /usr/bin/perl -w
      while (my $line = <STDIN>)

      { print $line; } --------- Input (test.txt) {code}

      test
      hi
      hello

      
      

      This infinite waiting happens randomly causing the job to fail with "Task attempt failed to report
      status for 605 seconds. Killing!".
      Same happens with 0.8 version too.

      Regards,
      Anitha

      Attachments

        1. PIG-2442-1.patch
          6 kB
          Daniel Dai
        2. PIG-2442_pig9.patch
          5 kB
          Jonathan Coveney

        Activity

          People

            xutingz xuting zhao
            anitharaju Anitha Raju
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: