Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: 1.2.0, 1.3.0
-
Component/s: DataSet API, Documentation
-
Labels:None
Description
the example of GroupReduce on sorted groups can't remove duplicate Strings in a DataSet.
need to add "prev=t"
such as:
val output = input.groupBy(0).sortGroup(1, Order.ASCENDING).reduceGroup {
(in, out: Collector[(Int, String)]) =>
var prev: (Int, String) = null
for (t <- in) {
if (prev == null || prev != t)
out.collect(t)
prev=t // this line is missing in the example
}
}