Pig
  1. Pig
  2. PIG-749

No attempt to check if 'flatten(group) as' has the same cardinality as 'group alias by'

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.3.0
    • Fix Version/s: 0.9.0
    • Component/s: grunt
    • Labels:
      None

      Description

      Pig script which does grouping for 3 columns and flattens as 4 columns works when in principle it should not and maybe fail as a front-end error.

      A = load 'groupcardinalitycheck.txt' using PigStorage() as (col1:chararray, col2:chararray, col3:int, col4:chararray);
      
      B = group A by (col1, col2, col3);
      
      C = foreach B generate
                 flatten(group) as (col1, col2, col3, col4),
                 SIZE(A) as frequency;
      
      dump C;
      
      

      ==========================================================================================
      Data
      ==========================================================================================
      hello CC 1 there
      hello YSO 2 out
      ouch CC 2 hey
      ==========================================================================================
      Result of the preceding script
      ==========================================================================================
      (ouch,CC,2,1L)
      (hello,CC,1,1L)
      (hello,YSO,2,1L)
      ==========================================================================================

        Activity

        Viraj Bhat created issue -
        Olga Natkovich made changes -
        Field Original Value New Value
        Fix Version/s 0.3.0 [ 12313785 ]
        Olga Natkovich made changes -
        Priority Major [ 3 ] Minor [ 4 ]
        Alan Gates made changes -
        Assignee Alan Gates [ alangates ]
        Fix Version/s 0.9.0 [ 12315191 ]
        Hide
        Daniel Dai added a comment -

        In the current code, it fails, but error message need to improve.

        Show
        Daniel Dai added a comment - In the current code, it fails, but error message need to improve.
        Hide
        Daniel Dai added a comment -

        Error message is enhanced by PIG-1737. Now the error message becomes:
        ERROR 1117: Cannot merge (group::col1:chararray,group::col2:chararray,group::col3:int) with user defined schema (col1:NULL,col2:NULL,col3:NULL,col4:NULL)

        Show
        Daniel Dai added a comment - Error message is enhanced by PIG-1737 . Now the error message becomes: ERROR 1117: Cannot merge (group::col1:chararray,group::col2:chararray,group::col3:int) with user defined schema (col1:NULL,col2:NULL,col3:NULL,col4:NULL)
        Daniel Dai made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Olga Natkovich made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Alan Gates
            Reporter:
            Viraj Bhat
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development