Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-696

Fatal error produced when malformed scalar types within complex type is converted to given type

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Instead of fatal error, the failed conversions should result in null values.
      Example -

      grunt > cat cbag3.dat

      {(asdf)} {(2344)} {(2344} {(323423423423434)} {(323423423423434L)} {(asdff)}

      grunt> A = load 'cbag3.dat' as (f1:bag

      {t:tuple(i:int)}

      ); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
      2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
      2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
      2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
      Details at logfile: /d1/tejas/pig_1236118410343.log

      tail /d1/tejas/pig_1236118410343.log
      Caused by: java.lang.ClassCastException
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)

      The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.

        Attachments

          Activity

            People

            • Assignee:
              alangates Alan Gates
              Reporter:
              thejas Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: