Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4059 Pig on Spark
  3. PIG-4438

Limit after sort does not work in spark mode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • spark-branch
    • spark
    • None

    Description

      when pig script executes "order" before "limit" in spark mode, the results will be wrong.
      cat testlimit.txt
      1 orange
      3 coconut
      5 grape
      6 pear
      2 apple
      4 mango

      testlimit.pig:
      a = load './testlimit.txt' as (x:int, y:chararray);
      b = order a by x;
      c = limit b 1;
      store c into './testlimit.out';

      the result:
      1 orange
      2 apple
      3 coconut
      4 mango
      5 grape
      6 pear

      the correct result should be:
      1 orange

      Attachments

        1. PIG-4438_2.patch
          1 kB
          liyunzhang
        2. PIG-4438_1.patch
          58 kB
          liyunzhang

        Issue Links

          Activity

            People

              kellyzly liyunzhang
              kellyzly liyunzhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: