[HBASE-15077] Support OffheapKV write in compaction with out copying data on heap

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: regionserver, Scanners
• Hadoop Flags: Reviewed
• Assignee: Anoop Sam John
• Reporter: Anoop Sam John
• Created: 07/Jan/16 16:19
• Updated: 17/Jun/22 05:25
• Resolved: 12/Jan/16 04:33

Description

HBASE-14832 is not enough to handle this. Doing the remaining needed here.

 if (cell instanceof ByteBufferedCell) {
890	      out.writeShort(rowLen);
891	      ByteBufferUtils.copyBufferToStream(out, ((ByteBufferedCell) cell).getRowByteBuffer(),
892	        ((ByteBufferedCell) cell).getRowPosition(), rowLen);
893	      out.writeByte(fLen);
894	      ByteBufferUtils.copyBufferToStream(out, ((ByteBufferedCell) cell).getFamilyByteBuffer(),
895	        ((ByteBufferedCell) cell).getFamilyPosition(), fLen);
896	      ByteBufferUtils.copyBufferToStream(out, ((ByteBufferedCell) cell).getQualifierByteBuffer(),
897	        ((ByteBufferedCell) cell).getQualifierPosition(), qLen);

We have done this but it is not really helping us!
In ByteBufferUtils#copyBufferToStream

public static void copyBufferToStream(OutputStream out, ByteBuffer in,
      int offset, int length) throws IOException {
    if (in.hasArray()) {
      out.write(in.array(), in.arrayOffset() + offset,
          length);
    } else {
      for (int i = 0; i < length; ++i) {
        out.write(toByte(in, offset + i));
      }
    }
  }
  

So for DBB it is so costly op writing byte by byte reading each to on heap.
Even if we use writeByteBuffer(OutputStream out, ByteBuffer b, int offset, int length), it won't help us as the underlying stream is a ByteArrayOutputStream and so we will end up in copying.

Issue Links

• is related to: HBASE-14832 Ensure write paths work with ByteBufferedCells in case of compaction