Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5354

Show fieldname and a line number for casting errors

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      One user had large number of casting in one foreach and wasn't able to tell which casting was causing this error from the error message.

      2018-08-03 17:45:17,716 [ERROR] [TezChild] |runtime.PigProcessor|: Encountered exception while processing:
      org.apache.pig.backend.executionengine.ExecException: ERROR 0: Exception while executing (Name: myprojection: Split - scope-41552 Operator Key: scope-41552): org.apache.pig.backend.executionengine.ExecException: ERROR 1074: Problem with formatting. Could not convert thisisnotanumber to Long.
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:315)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.getNextTuple(POSplit.java:208)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.runPipeline(POSplit.java:254)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.processPlan(POSplit.java:235)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POSplit.getNextTuple(POSplit.java:227)
          at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.runPipeline(PigProcessor.java:378)
          at org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor.run(PigProcessor.java:243)
          at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
          at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
          at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:422)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
          at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
          at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
          at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 1074: Problem with formatting. Could not convert thisisnotanumber to Long.
          at org.apache.pig.data.DataType.toLong(DataType.java:911)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.castBytesToLong(POCast.java:851)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNextLong(POCast.java:764)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:349)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getInputData(POForEach.java:531)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:398)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:343)
          at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:305)
          ... 19 more
      Caused by: java.lang.NumberFormatException: For input string: "thisisnotanumber"
          at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
          at java.lang.Long.parseLong(Long.java:589)
          at java.lang.Long.valueOf(Long.java:803)
          at org.apache.pig.data.DataType.toLong(DataType.java:880)
          ... 26 more
      

        Attachments

        1. pig-5354-v02.patch
          2 kB
          Koji Noguchi
        2. pig-5354-v01.patch
          1.0 kB
          Koji Noguchi

          Activity

            People

            • Assignee:
              knoguchi Koji Noguchi
              Reporter:
              knoguchi Koji Noguchi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: