Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
1.2.0, 1.3.0
-
None
Description
the example of GroupReduce on sorted groups can't remove duplicate Strings in a DataSet.
need to add "prev=t"
such as:
val output = input.groupBy(0).sortGroup(1, Order.ASCENDING).reduceGroup { (in, out: Collector[(Int, String)]) => var prev: (Int, String) = null for (t <- in) { if (prev == null || prev != t) out.collect(t) prev=t // this line is missing in the example } }