[AVRO-753] Java: Improve BinaryEncoder Performance

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: java
• Hadoop Flags: Incompatible change

Release Note

The Encoder API has several resulting changes:
    * Construction and configuration is handled by EncoderFactory. All
      Constructors are hidden, and Encoder.init(OutputStream) is removed.
    * Some Encoders previously did not buffer output. Users must call
      Encoder.flush() to ensure output is written unless the EncoderFactory
      method used to construct an instance explicitly states that the Encoder
      does not buffer output.

• Assignee: Scott Carey
• Reporter: Scott Carey
• Created: 01/Feb/11 09:07
• Updated: 12/Mar/11 00:32
• Resolved: 25/Feb/11 00:38

Description

BinaryEncoder has not had a performance improvement pass like BinaryDecoder did. It still mostly writes directly to the underlying OutputStream which is not optimal for performance. I like to use a rule that if you are writing to an OutputStream or reading from an InputStream in chunks smaller than 128 bytes, you have a performance problem.

Measurements indicate that optimizing BinaryEncoder yields a 2.5x to 6x performance improvement. The process is significantly simpler than BinaryDecoder because 'pushing' is easier than 'pulling' – and also because we do not need a 'direct' variant because BinaryEncoder already buffers sometimes.

Issue Links

• blocks: AVRO-769 Java: Align Decoder/Encoder APIs for consistency and long term stability
• is related to: AVRO-738 BinaryEncoder's init() function does not reset the OutputStream of the embedded SimpleByteWriter