Uploaded image for project: 'Crunch'
  1. Crunch
  2. CRUNCH-639

Writable Bytes does an unnecessary copy

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      In the Writable.bytes() Output MapFn, an unnecessary (I believe) copy of the incoming ByteBuffer occurs[0].

      Current:

      BytesWritable bw = new BytesWritable();
      bw.set(input.array(), input.arrayOffset(), input.limit()); <- copies the array
      

      Proposed:

      BytesWritable bw = new BytesWritable(input.array()); 
      

      [0]: https://github.com/apache/crunch/blob/apache-crunch-0.15.0/crunch-core/src/main/java/org/apache/crunch/types/writable/Writables.java#L271

        Attachments

          Activity

            People

            • Assignee:
              mkwhitacre Micah Whitacre
              Reporter:
              spatel89 Stephen Patel
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: