Pig
  1. Pig
  2. PIG-2428

In pig9, can't have limit(order by) without getting a null error

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0, 0.9.2
    • Fix Version/s: 0.9.2, 0.10.0, 0.11
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      a = load 'thing' as (x:int);
      b = group a by x;
      c = foreach b generate group as x, COUNT(a) as count;
      d = limit (order c by count DESC) 2000;
      describe d;
      

      This gives the following error:

      2011-12-13 13:56:32,144 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1005: No plan for d to describe

      In trunk, it ran without issue. Not sure what the difference is, but it'd be nice to patch 0.9.2 since a lot of people (including amazon!) are using pig 9 now.

      1. PIG-2428-0.patch
        0.8 kB
        Daniel Dai
      2. PIG-2428-1.patch
        3 kB
        Jonathan Coveney
      3. PIG-2428-2.patch
        3 kB
        Daniel Dai

        Activity

        Hide
        Daniel Dai added a comment -

        Seems some part of PIG-1926 does the trick. Attach that part.

        Show
        Daniel Dai added a comment - Seems some part of PIG-1926 does the trick. Attach that part.
        Hide
        Daniel Dai added a comment -

        Hi, Jonathan,
        Do you want me to add a test case? Or you want to work on that?

        Show
        Daniel Dai added a comment - Hi, Jonathan, Do you want me to add a test case? Or you want to work on that?
        Hide
        Jonathan Coveney added a comment -

        Daniel, I actually just added an (admittedly lightweight) test.

        I added it to TestParser since this error was coming up there, but it also tests that the sort/limit itself works. I wasn't sure where to put the test...originally I wanted it to be in TestOrderBy, but alas, those aren't included tests. I kept the test fairly contained so it could be moved easily.

        If you think there should be more, feel free to do so!

        Show
        Jonathan Coveney added a comment - Daniel, I actually just added an (admittedly lightweight) test. I added it to TestParser since this error was coming up there, but it also tests that the sort/limit itself works. I wasn't sure where to put the test...originally I wanted it to be in TestOrderBy, but alas, those aren't included tests. I kept the test fairly contained so it could be moved easily. If you think there should be more, feel free to do so!
        Hide
        Daniel Dai added a comment -

        The test case looks fine with me. I can commit it upon test passing.

        Show
        Daniel Dai added a comment - The test case looks fine with me. I can commit it upon test passing.
        Hide
        Dmitriy V. Ryaboy added a comment -

        just noticed Daniel attached the fix, and Jonathan only added the test... so assign as appropriate

        The test should be added to 9.2, 0.10 and trunk, not just 9.2

        Show
        Dmitriy V. Ryaboy added a comment - just noticed Daniel attached the fix, and Jonathan only added the test... so assign as appropriate The test should be added to 9.2, 0.10 and trunk, not just 9.2
        Hide
        Daniel Dai added a comment -

        PIG-2428-2.patch fix unit test failure.

        Show
        Daniel Dai added a comment - PIG-2428 -2.patch fix unit test failure.
        Hide
        Daniel Dai added a comment -

        Unit tests pass.

        test-patch:
        [exec] -1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
        [exec]
        [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

        I checked javadoc warning and it does not seems to be related.

        Patch committed to 0.9 branch (0.10/trunk already has this fix).

        Show
        Daniel Dai added a comment - Unit tests pass. test-patch: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. I checked javadoc warning and it does not seems to be related. Patch committed to 0.9 branch (0.10/trunk already has this fix).

          People

          • Assignee:
            Jonathan Coveney
            Reporter:
            Jonathan Coveney
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development