-
Type:
Bug
-
Status: Closed
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: 1.2.0, 1.3.0
-
Component/s: API / DataSet, Documentation
-
Labels:None
the example of GroupReduce on sorted groups can't remove duplicate Strings in a DataSet.
need to add "prev=t"
such as:
val output = input.groupBy(0).sortGroup(1, Order.ASCENDING).reduceGroup { (in, out: Collector[(Int, String)]) => var prev: (Int, String) = null for (t <- in) { if (prev == null || prev != t) out.collect(t) prev=t // this line is missing in the example } }