Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-2208

TableServers # processBatchOfRows - converts from List to [ ] - Expensive copy

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      With autoFlush to false and a large write buffer on HTable, when we write bulk puts - TableServer # processBatchOfRows , convert the input (List) to an [ ] , before sending down the wire.

      With a write buffer as large as 20 MB , that becomes an expensive copy when we do - list.toArray(new T[ ] ).

      May be - should we change the wire protocol to support List as well , and then revisit this to prevent the bulk copy ?

      Batch b = new Batch(this) {
              @Override
              int doCall(final List<Row> currentList, final byte [] row,
                final byte [] tableName)
              throws IOException, RuntimeException {
                *final Put [] puts = currentList.toArray(PUT_ARRAY_TYPE);*
                return getRegionServerWithRetries(new ServerCallable<Integer>(this.c,
                    tableName, row) {
                  public Integer call() throws IOException {
                    return server.put(location.getRegionInfo().getRegionName(), puts);
                  }
                });
              }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stack Michael Stack
                Reporter:
                kaykay.unique Karthik K
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: