[IMPALA-3708] Consider using DirectByteBuffers to pass serialized structs to frontend

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Henry Robinson
• Created: 09/Jun/16 19:03
• Updated: 19/Dec/17 20:08

Description

All of our JNI calls to the frontend use a byte[] to pass a serialized Thrift structure as an argument.

This causes at least one copy too many as we have to populate the byte array:

  uint8_t* buffer = NULL;
  uint32_t size = 0;
  RETURN_IF_ERROR(serializer.Serialize<T>(msg, &size, &buffer));

  /// create jbyteArray given buffer
  *serialized_msg = env->NewByteArray(size);
  RETURN_ERROR_IF_EXC(env);
  if (*serialized_msg == NULL) return Status("couldn't construct jbyteArray");
  env->SetByteArrayRegion(*serialized_msg, 0, size, reinterpret_cast<jbyte*>(buffer));

Instead, we should use a DirectByteBuffer to wrap the serialized buffer, which means we only have to pass the reference to the DBB around, rather than copy all the bytes into a Java object.

Issue Links

• depends upon: THRIFT-3432 Add a TByteBuffer transport to the Java library
• is related to: IMPALA-5690 Upgrade Thrift version to 0.9.3