Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-1861

The pig script stored in the Hadoop History logs is stored as a concatenated string without whitespace this causes problems when attempting to extract and execute the script

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: 0.8.1
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      a = load '$in' using com.yahoo.grid.sath.JobHistoryLoader() as
      (job:map[],maps:bag{},reducers:bag{},other:bag{},conf:map[]);

      The pig script stored in: conf#'pig.script' has the whitespace removed, this makes it difficult to extract and run the
      script. In particular, statements that terminate in ";" work correctly as
      "statement1;statement2;statement99" but statements that do not end in ";" result in
      "statement1statement2statement3" and it's difficult to parse the pig script and fix the concatenated string.

      There's also a problem with comments as in:

      /mycomment//more comments/ PIG_CODE //comment PIG_CODE
      On a side note, I also noticed that in many of the scripts the last statement is missing ";"

        Attachments

        1. PIG-1861.patch
          5 kB
          Richard Ding

          Activity

            People

            • Assignee:
              rding Richard Ding
              Reporter:
              araceli Araceli Henley
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: