Pig
  1. Pig
  2. PIG-696

Fatal error produced when malformed scalar types within complex type is converted to given type

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Instead of fatal error, the failed conversions should result in null values.
      Example -

      grunt > cat cbag3.dat

      {(asdf)} {(2344)} {(2344} {(323423423423434)} {(323423423423434L)} {(asdff)}

      grunt> A = load 'cbag3.dat' as (f1:bag

      {t:tuple(i:int)}

      ); B = foreach A generate flatten(f1); C = foreach B generate $0 + 1; dump C;
      2009-03-03 14:25:19,604 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
      2009-03-03 14:25:44,628 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Map reduce job failed
      2009-03-03 14:25:44,642 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2043: Unexpected error during execution.
      Details at logfile: /d1/tejas/pig_1236118410343.log

      tail /d1/tejas/pig_1236118410343.log
      Caused by: java.lang.ClassCastException
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.Add.getNext(Add.java:110)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:260)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:198)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:217)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:208)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)

      The 'conversion' of scalar types in complex types is happening in the physicaloperators, and not in the loaders. The expressions (such as Add in example) attempts to cast input to given type, and ClassCastException is thrown when conversion fails.

        Activity

        Hide
        Daniel Dai added a comment -

        Verified already fixed in trunk.

        Show
        Daniel Dai added a comment - Verified already fixed in trunk.

          People

          • Assignee:
            Alan Gates
            Reporter:
            Thejas M Nair
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development