Pig
  1. Pig
  2. PIG-230

auto ship broken is presence of define for another streaming operator.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0
    • Component/s: None
    • Labels:
      None

      Description

      The following works:

      A = load 'studenttab10k';
      B = foreach A generate $0, $1, $2;
      C = stream B through `perl PigStreaming.pl`;
      store C into 'out';

      But this one causes an error: Can't open perl script "PigStreaming.pl": No such file or directory.

      define CMD `perl PigStreamingDep.pl` input(stdin using PigDump) ship(':SCRIPTHOMEPATH:/PigStreamingDep.pl', ':SCRIPTHOMEPATH:/PigStreamingModule.pm');
      A = load 'studenttab10k';
      B = stream A through `perl PigStreaming.pl`;
      C = stream B through CMD as (name, age, gpa);
      D = foreach C generate name, age;
      store D into 'out';#,

        Activity

        Hide
        Arun C Murthy added a comment -

        Olga, was PigStreaming.pl on the PATH? Thanks.

        Show
        Arun C Murthy added a comment - Olga, was PigStreaming.pl on the PATH? Thanks.
        Hide
        Olga Natkovich added a comment -

        yes, it was. Both scripts ran in identical environment.

        Show
        Olga Natkovich added a comment - yes, it was. Both scripts ran in identical environment.
        Hide
        Arun C Murthy added a comment -

        Fixed POMapreduce to handle multiple ship/cache specs; updated TestStreaming to add this as a test-case ...

        Show
        Arun C Murthy added a comment - Fixed POMapreduce to handle multiple ship/cache specs; updated TestStreaming to add this as a test-case ...
        Hide
        Olga Natkovich added a comment -

        Arun, I have committed the patch, thanks. All my tests that used to fail now passing. If you could add a unit test for multiple cache options in addtion to multiple ship options, it would be great.

        You could just attache it to this bug. (I will keep it open till then.)

        Show
        Olga Natkovich added a comment - Arun, I have committed the patch, thanks. All my tests that used to fail now passing. If you could add a unit test for multiple cache options in addtion to multiple ship options, it would be great. You could just attache it to this bug. (I will keep it open till then.)
        Hide
        Olga Natkovich added a comment -

        latest patch for PIG-232 adds the test case

        Show
        Olga Natkovich added a comment - latest patch for PIG-232 adds the test case

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Olga Natkovich
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development