HBase
  1. HBase
  2. HBASE-2947

MultiIncrement/MultiAppend (MultiGet functionality for increments and appends)

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0
    • Component/s: Client, regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HBASE-1845 introduced MultiGet and other cross-row/cross-region batch operations. We should add a way to do that with increments.

      1. HBASE-2947-v1.patch
        2 kB
        Jonathan Gray
      2. 2947-v2.txt
        5 kB
        Lars Hofhansl
      3. 2947-final.txt
        0.9 kB
        Lars Hofhansl

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #65 (See https://builds.apache.org/job/HBase-TRUNK-security/65/)
          HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #65 (See https://builds.apache.org/job/HBase-TRUNK-security/65/ ) HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H) larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2612 (See https://builds.apache.org/job/HBase-TRUNK/2612/)
          HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2612 (See https://builds.apache.org/job/HBase-TRUNK/2612/ ) HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H) larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Hide
          Lars Hofhansl added a comment -

          Thanks for the review Stack!

          Show
          Lars Hofhansl added a comment - Thanks for the review Stack!
          Hide
          Lars Hofhansl added a comment -

          Patch that I committed (based on Stack's comment Append does not actually need to implement Writable, since Row already extends Writable).

          Show
          Lars Hofhansl added a comment - Patch that I committed (based on Stack's comment Append does not actually need to implement Writable, since Row already extends Writable).
          Hide
          Lars Hofhansl added a comment -

          Good point... I just checked, and Row extends WritableComparable, which in turn extends Writable... So should be good.
          Maybe than Append can be simplified to only implement Row and not also Writable.

          Show
          Lars Hofhansl added a comment - Good point... I just checked, and Row extends WritableComparable, which in turn extends Writable... So should be good. Maybe than Append can be simplified to only implement Row and not also Writable.
          Hide
          stack added a comment -

          Is this safe to do Lars?

          -public class Increment implements Writable {
          +public class Increment implements Row {
          

          Should you leave Writable in place?

          Else LGTM

          Show
          stack added a comment - Is this safe to do Lars? - public class Increment implements Writable { + public class Increment implements Row { Should you leave Writable in place? Else LGTM
          Hide
          Lars Hofhansl added a comment -

          Any objections?

          Show
          Lars Hofhansl added a comment - Any objections?
          Hide
          Lars Hofhansl added a comment - - edited

          I ran the four failing tests locally and they all pass.
          I would like to commit this today (the change itself is pretty uncontentious I think).

          Show
          Lars Hofhansl added a comment - - edited I ran the four failing tests locally and they all pass. I would like to commit this today (the change itself is pretty uncontentious I think).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509382/2947-v2.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -151 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
          org.apache.hadoop.hbase.mapreduce.TestImportTsv
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/663//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/663//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/663//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509382/2947-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -151 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/663//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/663//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/663//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          Patch for trunk, includes changes for Append and a test.

          Show
          Lars Hofhansl added a comment - Patch for trunk, includes changes for Append and a test.
          Hide
          Lars Hofhansl added a comment -

          While I am at it, I am also going to this for Append.

          Show
          Lars Hofhansl added a comment - While I am at it, I am also going to this for Append.
          Hide
          Lars Hofhansl added a comment -

          Cool... I'll add a test.

          Show
          Lars Hofhansl added a comment - Cool... I'll add a test.
          Hide
          Jonathan Gray added a comment -

          Not working on it but no reason not to commit that I recall.

          Show
          Jonathan Gray added a comment - Not working on it but no reason not to commit that I recall.
          Hide
          Lars Hofhansl added a comment -

          @Jon: Are you still working on this? Patch looks good to me and we have recently found some need for this.
          I'm happy to commit, unless there's opposition.

          Show
          Lars Hofhansl added a comment - @Jon: Are you still working on this? Patch looks good to me and we have recently found some need for this. I'm happy to commit, unless there's opposition.
          Hide
          Jonathan Gray added a comment -

          HBASE-2814 seems to only be about thrift. This is to make Increment a Row operation so it can be used with the existing MultiAction stuff.

          Show
          Jonathan Gray added a comment - HBASE-2814 seems to only be about thrift. This is to make Increment a Row operation so it can be used with the existing MultiAction stuff.
          Hide
          Andrew Purtell added a comment -

          (Partial) dup of HBASE-2814?

          Show
          Andrew Purtell added a comment - (Partial) dup of HBASE-2814 ?
          Hide
          Jonathan Gray added a comment -

          Will work on a unit test and javadoc additions/cleanup.

          Show
          Jonathan Gray added a comment - Will work on a unit test and javadoc additions/cleanup.
          Hide
          Jonathan Gray added a comment -

          Changes Increment to implement Row and adds the check for instanceof Increment in HRS.multi

          Show
          Jonathan Gray added a comment - Changes Increment to implement Row and adds the check for instanceof Increment in HRS.multi
          Hide
          Jonathan Gray added a comment -

          Most of what needs to be done is (trivially) make Increment implement Row.

          Show
          Jonathan Gray added a comment - Most of what needs to be done is (trivially) make Increment implement Row.

            People

            • Assignee:
              Lars Hofhansl
              Reporter:
              Jonathan Gray
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development