Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2286

Using COR function in Piggybank results in ERROR 2018: Internal error. Unable to introduce the combiner for optimization

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.1, 0.10.0
    • Component/s: impl, piggybank
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Usage of the COR function in a Pig script, results in an error. The "studenttab5" contains student, age and gpa separated by "tab".

      register /home/viraj/pig-svn/trunk/contrib/piggybank/java/piggybank.jar;
      A = LOAD '/user/viraj/studenttab5' AS (name, age:double,gpa:double);
      B = group A all;
      C = foreach B generate group, COR(A.a, A.b);
      dump C;
      

      2011-09-14 17:03:22,001 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000
      2011-09-14 17:03:22,088 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:9001
      2011-09-14 17:03:22,960 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: GROUP_BY
      2011-09-14 17:03:23,168 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
      2011-09-14 17:03:23,179 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer - Choosing to move algebraic foreach to combiner
      2011-09-14 17:03:23,186 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2018: Internal error. Unable to introduce the combiner for optimization.

      Viraj

        Attachments

        1. PIG-2286-2.patch
          6 kB
          Daniel Dai
        2. PIG-2286-1.patch
          5 kB
          Daniel Dai

          Activity

            People

            • Assignee:
              daijy Daniel Dai
              Reporter:
              viraj Viraj Bhat
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: