Pig
  1. Pig
  2. PIG-1241

Accumulator is turned on when a map is used with a non-accumulative UDF

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Exception is thrown for a script like the following:

      register /homes/yinghe/owl/string.jar;
      a = load 'a.txt' as (id, url);
      b = group a by (id, url);
      c = foreach b generate COUNT(a), (CHARARRAY) string.URLPARSE(group.url)#'url';
      dump c;

      In this query, URLPARSE() is not accumulative, and it returns a map.

      The accumulator optimizer failed to check UDF in this case, and tries to run the job in accumulative mode. ClassCastException is thrown when trying to cast UDF into Accumulator interface.

        Activity

          People

          • Assignee:
            Ying He
            Reporter:
            Ying He
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development