Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2224

Incorrect arity test in AstValidator.g with ALL and column-based grouping condition together in cogroup

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.9.0
    • None
    • grunt
    • Suse Linux 9/MacOS(10.7)

    • Patch Available
    • cogroup, grunt, astvalidator, arity, all

    Description

      When ALL and column-based grouping condition are used together in COGROUP, the arity test in AstValidator.g (line 242) incorrectly sets the arity and causes exception. For example, assume we have the follow two relations:

      a = load 'A' as (col_a_0, col_a_1);
      b = load 'B' as (col_b_0, col_b_1);

      The following statement will throw an invalidation error:

      c = cogroup a by col_a_0, b ALL;

      It is because when processing a:col_a_0, the arity is set to 1; then when processing b:ALL, due to the null value in join_group_by_clause will emit arity 0 for the second relation, and arity test fails.

      Reversing the two relations will be a work-around for this error:

      c = cogroup b ALL, a by col_a_0;

      However it is a lucky shot: when processing b:ALL, since join_group_by_clause is null, arity is still 0; then when processing a:col_a_0, arity will be initialized so no arity test is done in this case (so it passes).

      The main reason is the omission of the consideration on ALL keyword during the arity test. I attached a patch to fix this, by separating the arity test for both join_group_by_clause and ALL. The patch is tested locally and it works.

      Attachments

        1. pig-2224.diff.patch
          1 kB
          JArod Wen
        2. pig-2224.diff.patch
          2 kB
          JArod Wen

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            jarodwen JArod Wen

            Dates

              Created:
              Updated:

              Slack

                Issue deployment