Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2498

e2e tests failing in some cases due to incorrect unix sort args


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.9.2
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:


      Some e2e tests are failing for me against 23 due to what I think are incorrect arguments to unix sort. For example in Order_6:

      			'num' => 6,
      			'pig' => q\a = load ':INPATH:/singlefile/studenttab10k';
      c = order a by $0;
      store c into ':OUTPATH:';\,
      			'sortArgs' => ['-t', '	', '+0', '-1'],

      The pig job is sorting by the first column, however unix sort is being told to sort by the first and second columns.

      From the gnu sort manual (specifically pos2 is inclusive): http://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html

      '-k pos1[,pos2]'
      Specify a sort field that consists of the part of the line between pos1 and pos2 (or the end of the line, if pos2 is omitted), inclusive.
      On older systems, sort supports an obsolete origin-zero syntax '+pos1 [-pos2]' for specifying sort keys. The obsolete sequence 'sort +a.x -b.y' is equivalent to 'sort -k a+1.x+1,b' if y is '0' or absent, otherwise it is equivalent to 'sort -k a+1.x+1,b+1.y'.

      I verified this by running the sort manually with +0 -1 and +0 -0, in the first case it fails, in the second case it passes.


        1. PIG-2498.patch
          10 kB
          Patrick Hunt

          Issue Links



              • Assignee:
                phunt Patrick Hunt
                phunt Patrick Hunt
              • Votes:
                0 Vote for this issue
                5 Start watching this issue


                • Created: