[CRUNCH-525] The ExtractKeyFn is has an incorrect scale factor

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Core
• Assignee: Josh Wills
• Reporter: Stephen Patel
• Created: 21/May/15 21:51
• Updated: 06/Aug/15 04:39
• Resolved: 22/May/15 18:41

Description

The ExtractKeyFn[0] used by the by[1] method of the PCollectionImpl is using the default scale factor for a MapFn (1.0). It should be using 1.0 + the scale factor of the wrapped MapFn, in order to be accurate.

[0]: https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/fn/ExtractKeyFn.java
[1]: https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/impl/dist/collect/PCollectionImpl.java#L270