Pig
  1. Pig
  2. PIG-2159

New logical plan uses incorrect class for SUM causing for ClassCastException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The below is my script;

      A = load 'input1' using PigStorage(',')  as (f1:int,f2:int,f3:int,f4:long,f5:double);
      B = load 'input2' using PigStorage(',')  as (f1:int,f2:int,f3:int,f4:long,f5:double);
      C = load 'input_Main' using PigStorage(',')  as (f1:int,f2:int,f3:int);
      U = UNION ONSCHEMA A,B;
      J = join C by (f1,f2,f3) LEFT OUTER, U by (f1,f2,f3);
      Porj = foreach J generate C::f1 as f1 ,C::f2 as f2,C::f3 as f3,U::f4 as f4,U::f5 as f5;
      G = GROUP Porj by (f1,f2,f3,f5);
      Final = foreach G generate SUM(Porj.f4) as total;
      dump Final;
      

      The script fails at while computing the sum with class cast exception.
      Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Double
      at org.apache.pig.builtin.DoubleSum$Initial.exec(DoubleSum.java:82)
      ... 19 more

      This is clearly a bug in the logical plan created in 0.9. The sum operation should have processed using org.apache.pig.builtin.LongSum, but instead 0.9 logical plan have used org.apache.pig.builtin.DoubleSum which is meant for sum of doubles. And hence the ClassCastException.

      The same script works fine with Pig 0.8.

      1. PIG-2159-2.patch
        6 kB
        Daniel Dai
      2. PIG-2159-1.patch
        5 kB
        Daniel Dai

        Activity

        Vivek Padmanabhan created issue -
        Dmitriy V. Ryaboy made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Blocker [ 1 ]
        Daniel Dai made changes -
        Attachment PIG-2159-1.patch [ 12486523 ]
        Daniel Dai made changes -
        Attachment PIG-2159-2.patch [ 12486537 ]
        Daniel Dai made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Daniel Dai made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Assignee Daniel Dai [ daijy ]
        Resolution Fixed [ 1 ]
        Olga Natkovich made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Vivek Padmanabhan
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development