Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3412

jsonstorage breaks when tuple does not have as many columns as schema

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.11
    • 0.12.0
    • None
    • None
    • Patch Available

    Description

      Noticed this error when doing something like
      A = flatten(STRSPLIT($0, ',', 3)) AS (col1:chararray, col2:chararray, col3:chararray);
      STORE A INTO 'foo' USING JsonStorage();

      If the string being split doesn't generate 3 columns, then JsonStorage errors out with an index exception. This is because it tries to read the fields of the tuple passed to it or not. See JsonStorage, line 148.

      MY patch checks the length of the tuple. If any schema column positions are past the length of the tuple, it fills in null.

      Attachments

        1. jsonStoragePatch.patch
          1 kB
          Adam Silberstein

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            aesilberstein Adam Silberstein
            aesilberstein Adam Silberstein
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment