[PIG-5212] SkewedJoin_6 is failing on Spark

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: liyunzhang
• Reporter: Nándor Kollár
• Created: 04/Apr/17 07:58
• Updated: 21/Jun/17 09:18
• Resolved: 19/Apr/17 08:18

Description

result are different:

diff <(head -20 SkewedJoin_6_benchmark.out/out_sorted) <(head -20 SkewedJoin_6.out/out_sorted)
< alice allen	19	1.930	alice allen	27	1.950
< alice allen	19	1.930	alice allen	34	1.230
< alice allen	19	1.930	alice allen	36	2.270
< alice allen	19	1.930	alice allen	38	0.810
< alice allen	19	1.930	alice allen	38	1.800
< alice allen	19	1.930	alice allen	42	2.460
< alice allen	19	1.930	alice allen	43	0.880
< alice allen	19	1.930	alice allen	45	2.800
< alice allen	19	1.930	alice allen	46	3.970
< alice allen	19	1.930	alice allen	51	1.080
< alice allen	19	1.930	alice allen	68	3.390
< alice allen	19	1.930	alice allen	68	3.510
< alice allen	19	1.930	alice allen	72	1.750
< alice allen	19	1.930	alice allen	72	3.630
< alice allen	19	1.930	alice allen	74	0.020
< alice allen	19	1.930	alice allen	74	2.400
< alice allen	19	1.930	alice allen	77	2.520
< alice allen	20	2.470	alice allen	27	1.950
< alice allen	20	2.470	alice allen	34	1.230
< alice allen	20	2.470	alice allen	36	2.270
---
> alice allen	27	1.950	alice allen	19	1.930
> alice allen	27	1.950	alice allen	20	2.470
> alice allen	27	1.950	alice allen	27	1.950
> alice allen	27	1.950	alice allen	34	1.230
> alice allen	27	1.950	alice allen	36	2.270
> alice allen	27	1.950	alice allen	38	0.810
> alice allen	27	1.950	alice allen	38	1.800
> alice allen	27	1.950	alice allen	42	2.460
> alice allen	27	1.950	alice allen	43	0.880
> alice allen	27	1.950	alice allen	45	2.800
> alice allen	27	1.950	alice allen	46	3.970
> alice allen	27	1.950	alice allen	51	1.080
> alice allen	27	1.950	alice allen	68	3.390
> alice allen	27	1.950	alice allen	68	3.510
> alice allen	27	1.950	alice allen	72	1.750
> alice allen	27	1.950	alice allen	72	3.630
> alice allen	27	1.950	alice allen	74	0.020
> alice allen	27	1.950	alice allen	74	2.400
> alice allen	27	1.950	alice allen	77	2.520
> alice allen	34	1.230	alice allen	19	1.930

It looks like the two tables are in wrong order, columns from 'a' should come first, then columns from 'b'. In spark mode this is inverted.