Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3412

jsonstorage breaks when tuple does not have as many columns as schema

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Noticed this error when doing something like
      A = flatten(STRSPLIT($0, ',', 3)) AS (col1:chararray, col2:chararray, col3:chararray);
      STORE A INTO 'foo' USING JsonStorage();

      If the string being split doesn't generate 3 columns, then JsonStorage errors out with an index exception. This is because it tries to read the fields of the tuple passed to it or not. See JsonStorage, line 148.

      MY patch checks the length of the tuple. If any schema column positions are past the length of the tuple, it fills in null.

        Attachments

        1. jsonStoragePatch.patch
          1 kB
          Adam Silberstein

          Activity

            People

            • Assignee:
              aesilberstein Adam Silberstein
              Reporter:
              aesilberstein Adam Silberstein
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: