Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-525

The ExtractKeyFn is has an incorrect scale factor

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: Core
    • Labels:
      None

      Description

      The ExtractKeyFn[0] used by the by[1] method of the PCollectionImpl is using the default scale factor for a MapFn (1.0). It should be using 1.0 + the scale factor of the wrapped MapFn, in order to be accurate.

      [0]: https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/fn/ExtractKeyFn.java
      [1]: https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/impl/dist/collect/PCollectionImpl.java#L270

        Attachments

        1. CRUNCH-525.patch
          2 kB
          Josh Wills
        2. CRUNCH-525b.patch
          2 kB
          Josh Wills

          Activity

            People

            • Assignee:
              jwills Josh Wills
              Reporter:
              spatel89 Stephen Patel
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: