[SPARK-4805] BlockTransferMessage.toByteArray() trips assertion

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Component/s: Shuffle, Spark Core
• Assignee: Sean R. Owen
• Reporter: Sean R. Owen
• Created: 09/Dec/14 22:49
• Updated: 17/May/20 18:30
• Resolved: 10/Dec/14 00:39

Description

The ByteBuf below is allocated to have enough room to encode just the message, but a type byte is written too. The buffer resizes and is left with capacity, which is a minor waste. But when assertions are on, it fails the assertion below. This is triggered by unit tests, but the Java test for this wasn't running. You can verify that a simple "+1" in the allocated size fixes it.

  public byte[] toByteArray() {
    ByteBuf buf = Unpooled.buffer(encodedLength());
    buf.writeByte(type().id);
    encode(buf);
    assert buf.writableBytes() == 0 : "Writable bytes remain: " + buf.writableBytes();
    return buf.array();
  }

ilikerps

Issue Links

• links to: [Github] Pull Request #3650 (srowen)