Pig
  1. Pig
  2. PIG-698

Simple join fails on records not loaded with schema

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7.0
    • Component/s: impl
    • Labels:
      None
    • Environment:

      Yahoo! clusters.

      Description

      Joins can fail with an out-of-bounds access to fields that are not referenced in the script when records without schema (including all variable-length records) are involved.
      Example by Ben Reed:
      i1:
      1 c D E
      1 a B

      i2:
      0
      0 Q
      1 x z
      1 a b c

      i1 = load 'i1';
      i2 = load 'i2';
      j = join i1 by $0, i2 by $0;
      dump j

        Activity

        Alan Gates made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Alan Gates made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.7.0 [ 12314397 ]
        Fix Version/s 0.9.0 [ 12315191 ]
        Resolution Fixed [ 1 ]
        Hide
        Alan Gates added a comment -

        Tested this against version 0.7 and it works fine.

        Show
        Alan Gates added a comment - Tested this against version 0.7 and it works fine.
        Olga Natkovich made changes -
        Fix Version/s 0.9.0 [ 12315191 ]
        Peter Arthur Ciccolo made changes -
        Field Original Value New Value
        Summary Simple join fails on variable-length records Simple join fails on records not loaded with schema
        Description Joins can fail with an out-of-bounds access to fields that are not referenced in the script when variable-length
        records are involved.
        Example by Ben Reed:
        i1:
        1 c D E
        1 a B

        i2:
        0
        0 Q
        1 x z
        1 a b c


        i1 = load 'i1';
        i2 = load 'i2';
        j = join i1 by $0, i2 by $0;
        dump j
        Joins can fail with an out-of-bounds access to fields that are not referenced in the script when records without schema (including all variable-length records) are involved.
        Example by Ben Reed:
        i1:
        1 c D E
        1 a B

        i2:
        0
        0 Q
        1 x z
        1 a b c


        i1 = load 'i1';
        i2 = load 'i2';
        j = join i1 by $0, i2 by $0;
        dump j
        Peter Arthur Ciccolo created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Peter Arthur Ciccolo
          • Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development