XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: spark-branch
    • Component/s: spark
    • Labels:
      None

      Description

      Union_3 e2e test is failing on spark branch. If I execute on spark branch in MR mode (test-e2e instead of test-e2e-spark), the test is passing.

      It fails while comparing the output of the second foreach (the output of the first foreach is fine):
      RESULTS DIFFER: vimdiff /var/lib/hadoop-hdfs/pig-spark/pig/test/e2e/pig/testdist/./out/pigtest/hdfs/hdfs-1487068760-nightly.conf/Union_3.out/2/out_sorted ./out/pigtest/hdfs/hdfs-1487068760-nightly.conf/Union_3_benchmark.out/2/out_sorted

      The first 3 lines:

      diff <(head -n 3 ./Union_3_benchmark.out/2/out_sorted) <(head -n 3 ./Union_3.out/2/out_sorted)
      1,3c1,3
      < {(alice allen,19,1.93)}
      < {(alice brown,18,2.35)}
      < {(alice carson,19,1.88)}
      ---
      > {(alice allen,77,3.51)}
      > {(alice brown,77,0.74)}
      > {(alice carson,74,1.73)}
      

      Old Pig is path is ../../../.., so the benchmarks were created with the same Pig in MR mode.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                nkollar Nándor Kollár
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: