Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-51

Combiner gives wrong result in the presence of flattening

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.1.0
    • None
    • None
    • Patch Available

    Description

      If you do something like

      a = load ... as (f1,f2,f3);
      b = group a by (f1,f2);
      c = foreach b generate flatten(group), SUM(a.f3);

      The reduce side refers to field number expecting data will not have been flattened yet. But if the combiner kicks in, it already flattens the group, leading to column references being wrong.

      Attachments

        1. combiner-flatten.patch
          4 kB
          Utkarsh Srivastava

        Activity

          People

            utkarsh Utkarsh Srivastava
            utkarsh Utkarsh Srivastava
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: