Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5058

Pig 0.16 e2e Types_Order tests failed with Sort check failed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • e2e harness
    • None
    • CentOS 7.1
      Ubuntu 14.04
      Mac OS 10.11.5 (15F34)

    Description

      Env: core 5.2, centOS 7.1
      pig: pig-0.16

      Pig 0.16 e2e tests Types_Order_1,2,3,4,11,12,13,14,15,16 failed because of Sort check failed.

      test-base:
      [exec] =========================================================
      [exec] LOGGING RESULTS TO /pig/pig-0.16/test/e2e/pig/testdist/out/log/test_harnesss_1478952742
      [exec] =========================================================
      [exec] Results so far, PASSED: 0 FAILED: 1 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 0 FAILED: 2 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 0 FAILED: 3 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 0 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 1 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 2 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 3 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 4 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 5 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 5 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 6 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 7 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 8 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 9 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Results so far, PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
      [exec] Final results , PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0

      BUILD FAILED

      TEST: Types_Order_1
      sort ./out/pigtest-1478952742-nightly.conf/Types_Order_1_benchmark.out/out_original
      test cksum: 1595601925 208685
      benchmark cksum: 1595601925 208685
      Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/1478952742-nightly.conf/Types_Order_1.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original:27: disorder: 18
      Sort check failed

      TEST: Types_Order_2
      Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original:23: disorder: zach young 3.34
      Sort check failed

      TEST: Types_Order_3
      Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original:27: disorder: 18
      Sort check failed

      TEST: Types_Order_4
      Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original:23: disorder: zach young 3.34
      Sort check failed

      TEST: Types_Order_11
      Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original:731: disorder: oscar underhill 58 0.1
      Sort check failed

      TEST: Types_Order_12
      Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original:4: disorder: irene johnson 61 3.99
      Sort check failed

      TEST: Types_Order_13
      Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original:731: disorder: oscar underhill 58 0.1
      Sort check failed

      TEST: Types_Order_14
      Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original:4: disorder: irene johnson 61 3.99
      Sort check failed

      TEST: Types_Order_15
      Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original:27: disorder: 18
      Sort check failed

      TEST: Types_Order_16
      Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original
      /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original:23: disorder: zach young 3.34
      Sort check failed

      REASON:
      In the process of investigation it became clear the following: when we store data in pig, we assign them the data type. This leads to the fact that the file size changed (probably because of the removal of extra delimiters). As a result of subsequent actions sorting is carried out already with the changed data (different file size). In bash (on Cent OS, Ubuntu and Mac OS), in turn, we do not have the ability to work with data types, and when we perform any sorting, we make them with the same "raw data". As a result, the check at the end of the test is incorrect because the files that are compared have different size.
      We also have different results during sorting on Cent OS/Ubuntu and Mac OS. All these inaccuracies lead to failure in sort check.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Konstantin_Harasov Konstantin Harasov
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: