[HDDS-4471] GrpcOutputStream length can overflow

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.1.0
• Component/s: Ozone Datanode
• Labels: pull-request-available
• Assignee: YI-CHEN WANG
• Reporter: Wei-Chiu Chuang
• Created: 17/Nov/20 07:00
• Updated: 01/Dec/20 02:17
• Resolved: 01/Dec/20 02:17

Description

DN says it sent negative bytes of container data to destination.

2020-11-16 22:07:26,445 INFO org.apache.hadoop.ozone.container.replication.GrpcReplicationService: Streaming container data (982134) to other datanode
2020-11-16 22:16:37,693 INFO org.apache.hadoop.ozone.container.replication.GrpcOutputStream: Sent -2033242913 bytes for container 982134

Looks like there was an overflow. The writtenBytes shouldn't be an int, because the container is as big as 5GB, easily overflow a signed int.

GrpcOutputStream.java

  private int writtenBytes;

...
      CopyContainerResponseProto response =
          CopyContainerResponseProto.newBuilder()
              .setContainerID(containerId)
              .setData(data)
              .setEof(eof)
              .setReadOffset(writtenBytes)
              .setLen(length)
              .build();
      responseObserver.onNext(response);
      writtenBytes += length;

Looking at the source code, I think we can change it to long without breaking serialization compatibility because setReadOffset() expects a long.

Looks like a minor issue to me. The other side doesn't use the read offset, so even though it can overflow, there's no real impact. Still, we should fix it.

Issue Links

• links to: GitHub Pull Request #1617