[HIVE-3823] Performance issue while retrieving the Result objects in HiveHBaseTableInputFormat

• Type: Improvement
• Status: Open
• Priority: Trivial
• Resolution: Unresolved
• Affects Version/s: 0.9.0, 0.9.1, 0.10.0
• Component/s: HBase Handler
• Assignee: Unassigned
• Reporter: Cheng Hao
• Created: 20/Dec/12 01:09
• Updated: 02/May/13 02:29

Description

In HiveHBaseTableInputFormat.java, the Result objects retrieving has performance issue.

HiveHBaseTableInputFormat.java

      @Override
      public boolean next(ImmutableBytesWritable rowKey, Result value) throws IOException {

        boolean next = false;

        try {
          next = recordReader.nextKeyValue();

          if (next) {
            rowKey.set(recordReader.getCurrentValue().getRow());
            // performance issue here, as the copyWritable
            // is Serialization - Bytes Copying - Deserialization.
            Writables.copyWritable(recordReader.getCurrentValue(), value);
          }
        } catch (InterruptedException e) {
          throw new IOException(e);
        }

        return next;
      }

In HBASE 0.94.4 & 0.96.0, the Result provides a new method "copyFrom(Result from)", will solve the issue.

See HBASE-7381

Issue Links

• depends upon: HBASE-7381 Lightweight data transfer for Class Result