Pig
  1. Pig
  2. PIG-125

improve exception handling and expressivness around tuple field access

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.1.0
    • Component/s: impl
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      Stumbled over the case that i'm accessing fields in a tuple which type are not as i expected. The stack trace in one case looked as follow:

      Exception in thread "main" java.lang.RuntimeException: execution failed
      	at com.my.Executor.run(Executor.java:284)
      Caused by: java.io.IOException: Unable to store alias C
      	at org.apache.pig.impl.util.WrappedIOException.wrap(WrappedIOException.java:16)
      	at org.apache.pig.PigServer.store(PigServer.java:335)
      	at org.apache.pig.PigServer.store(PigServer.java:317)
      	at com.my.Executor.run(Executor.java:280)
      	... 2 more
      Caused by: org.apache.pig.backend.executionengine.ExecException
      	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:137)
      	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:32)
      	at org.apache.pig.PigServer.store(PigServer.java:332)
      	... 4 more
      Caused by: java.io.IOException: Incompatible type for request getAtomField().
      	at org.apache.pig.data.Tuple.getAtomField(Tuple.java:177)
      	at com.my.DatabaseStoreFunc.putNext(DatabaseStoreFunc.java:83)
      	at org.apache.pig.impl.io.PigFile.store(PigFile.java:64)
      	at org.apache.pig.backend.local.executionengine.POStore.getNext(POStore.java:105)
      	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:130)
      	... 6 more
      

      The exception message and the stacktrace gave me a clue what kind of problem i was facing. But to know what exactly happened i needed to debug (or temporarily add some system-outs).
      Looking at the code (of Tuple class) i think the exception-information can be improved easily (add index and actual field type information) .
      Also it seems that there is some space for simplifying the exception handling.

      1. PIG-125.patch
        11 kB
        Johannes Zillmann

        Activity

        Hide
        Johannes Zillmann added a comment -

        I've uploaded a patch which contains both, an enrichment of the exeption message with more information and a slightly simpliefied exception handling.
        For the latter i simply replaced the throwing of IOExceptions with a throwing of RuntimeExceptions (IllegalArgumentException). This elimintated some "catch IOException, throw new RuntimeException(ioEx)" in other places.

        The stacktrace with this patch would look like

        Exception in thread "main" java.lang.RuntimeException: running mmp3 failed
        	at com.my.Executor.run(Executor.java:284)
        Caused by: java.io.IOException: Unable to store alias C
        	at org.apache.pig.impl.util.WrappedIOException.wrap(WrappedIOException.java:16)
        	at org.apache.pig.PigServer.store(PigServer.java:335)
        	at org.apache.pig.PigServer.store(PigServer.java:317)
        	at my.Executor.run(Executor.java:280)
        	... 2 more
        Caused by: org.apache.pig.backend.executionengine.ExecException
        	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:137)
        	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:1)
        	at org.apache.pig.PigServer.store(PigServer.java:332)
        	... 4 more
        Caused by: java.lang.IllegalArgumentException: Requested atom field at index 16 but was 'org.apache.pig.data.DefaultDataBag' in tuple: (VOICE, 6502815167, 6502815167, 1187930860987, 1187930870658, 1187913600000, 1187913600000, 17260987, 17270658, 1187930860987, 1187930870658, 9671, 21, 006db2cbeb55ce9d3c7de46261be068e, 0, BLOCK, {}, 9671.0)
        	at org.apache.pig.data.Tuple.newTupleAccessException(Tuple.java:183)
        	at org.apache.pig.data.Tuple.getAtomField(Tuple.java:178)
        	at com.my.DatabaseStoreFunc.putNext(DatabaseStoreFunc.java:83)
        	at org.apache.pig.impl.io.PigFile.store(PigFile.java:64)
        	at org.apache.pig.backend.local.executionengine.POStore.getNext(POStore.java:105)
        	at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:130)
        	... 6 more
        
        Show
        Johannes Zillmann added a comment - I've uploaded a patch which contains both, an enrichment of the exeption message with more information and a slightly simpliefied exception handling. For the latter i simply replaced the throwing of IOExceptions with a throwing of RuntimeExceptions (IllegalArgumentException). This elimintated some "catch IOException, throw new RuntimeException(ioEx)" in other places. The stacktrace with this patch would look like Exception in thread "main" java.lang.RuntimeException: running mmp3 failed at com.my.Executor.run(Executor.java:284) Caused by: java.io.IOException: Unable to store alias C at org.apache.pig.impl.util.WrappedIOException.wrap(WrappedIOException.java:16) at org.apache.pig.PigServer.store(PigServer.java:335) at org.apache.pig.PigServer.store(PigServer.java:317) at my.Executor.run(Executor.java:280) ... 2 more Caused by: org.apache.pig.backend.executionengine.ExecException at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:137) at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:1) at org.apache.pig.PigServer.store(PigServer.java:332) ... 4 more Caused by: java.lang.IllegalArgumentException: Requested atom field at index 16 but was 'org.apache.pig.data.DefaultDataBag' in tuple: (VOICE, 6502815167, 6502815167, 1187930860987, 1187930870658, 1187913600000, 1187913600000, 17260987, 17270658, 1187930860987, 1187930870658, 9671, 21, 006db2cbeb55ce9d3c7de46261be068e, 0, BLOCK, {}, 9671.0) at org.apache.pig.data.Tuple.newTupleAccessException(Tuple.java:183) at org.apache.pig.data.Tuple.getAtomField(Tuple.java:178) at com.my.DatabaseStoreFunc.putNext(DatabaseStoreFunc.java:83) at org.apache.pig.impl.io.PigFile.store(PigFile.java:64) at org.apache.pig.backend.local.executionengine.POStore.getNext(POStore.java:105) at org.apache.pig.backend.local.executionengine.LocalExecutionEngine.execute(LocalExecutionEngine.java:130) ... 6 more
        Hide
        Alan Gates added a comment -

        Fix checked in at revision 632427. Thanks Johannes.

        Show
        Alan Gates added a comment - Fix checked in at revision 632427. Thanks Johannes.

          People

          • Assignee:
            Johannes Zillmann
            Reporter:
            Johannes Zillmann
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development