Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.6.0
    • Fix Version/s: None
    • Component/s: impl
    • Labels:
      None

      Description

      For the purpose of easy debugging, I would be nice to find out where my warnings are coming from is in the pig script.

      The only known process is to comment out lines in the Pig script and see if these warnings go away.

      2010-01-13 21:34:13,697 [main] WARN org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_MAP 2 time(s) line 22
      2010-01-13 21:34:13,698 [main] WARN org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_LONG 2 time(s) line 23
      2010-01-13 21:34:13,698 [main] WARN org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_BAG 1 time(s). line 26

      I think this may need us to keep track of the line numbers of the Pig script (via out javacc parser) and maintain it in the logical and physical plan.

      It would help users in debugging simple errors/warning related to casting.

      Is this enhancement listed in the http://wiki.apache.org/pig/PigJournal?

      Do we need to change the parser to something other than javacc to make this task simpler?

      "Standardize on Parser and Scanner Technology"

      Viraj

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          This will involve recording line numbers (and possibly more metadata) from parser to logical layer, then to physical layer and then to backend and then back in case of exceptions. This has been discussed before in some detail in PIG-908. Linking it against that.

          Show
          Ashutosh Chauhan added a comment - This will involve recording line numbers (and possibly more metadata) from parser to logical layer, then to physical layer and then to backend and then back in case of exceptions. This has been discussed before in some detail in PIG-908 . Linking it against that.
          Hide
          Viraj Bhat added a comment -

          Linking this issue to PIG:908, though I am not sure if we have to wait for PIG:908 to fix this one.

          Show
          Viraj Bhat added a comment - Linking this issue to PIG:908, though I am not sure if we have to wait for PIG:908 to fix this one.
          Hide
          Viraj Bhat added a comment -

          Which release will PIG:908 be fixed?

          Does it guarantee that if we fix PIG:908, then this issue will be solved?

          Show
          Viraj Bhat added a comment - Which release will PIG:908 be fixed? Does it guarantee that if we fix PIG:908, then this issue will be solved?
          Hide
          Richard Ding added a comment -

          By default, the Pig property "aggregate.warning" is set to true. So in the above example, you only get aggregated warning messages, not the original detailed warning messages. You can turn off "aggregate.warning" by the command line switch -w. The detailed warning messages contain more information. Here is an example:

          [main] WARN  org.apache.pig.PigServer - int is implicitly cast to float under LOAdd Operator
          [main] WARN  org.apache.pig.PigServer - long is implicitly cast to float under LOAdd Operator 
          

          instead of

          [main] WARN  org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_FLOAT 2 time(s).
          
          Show
          Richard Ding added a comment - By default, the Pig property "aggregate.warning" is set to true. So in the above example, you only get aggregated warning messages, not the original detailed warning messages. You can turn off "aggregate.warning" by the command line switch -w. The detailed warning messages contain more information. Here is an example: [main] WARN org.apache.pig.PigServer - int is implicitly cast to float under LOAdd Operator [main] WARN org.apache.pig.PigServer - long is implicitly cast to float under LOAdd Operator instead of [main] WARN org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_FLOAT 2 time(s).
          Hide
          Viraj Bhat added a comment -

          Richard thanks for suggesting a workaround. The error message is definitely more verbose than the original one.

          At least in one way the user can know as to where the cast is an issue in the, maybe in some addition taking place in the script.

          This Jira was originally created as task to correlate exactly on which line "int is implicitly cast to float", which I believe is hard to do in the current parser as we do not keep track of line number.

          Viraj

          Show
          Viraj Bhat added a comment - Richard thanks for suggesting a workaround. The error message is definitely more verbose than the original one. At least in one way the user can know as to where the cast is an issue in the, maybe in some addition taking place in the script. This Jira was originally created as task to correlate exactly on which line "int is implicitly cast to float", which I believe is hard to do in the current parser as we do not keep track of line number. Viraj
          Hide
          Viraj Bhat added a comment -

          Changing release to Pig 0.9.

          Show
          Viraj Bhat added a comment - Changing release to Pig 0.9.
          Hide
          Olga Natkovich added a comment -

          delaying till 0.10

          Show
          Olga Natkovich added a comment - delaying till 0.10
          Hide
          Amol Kekre added a comment -

          we are taking it up in .10 patch

          Show
          Amol Kekre added a comment - we are taking it up in .10 patch

            People

            • Assignee:
              Unassigned
              Reporter:
              Viraj Bhat
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development