HBase
  1. HBase
  2. HBASE-2947

MultiIncrement/MultiAppend (MultiGet functionality for increments and appends)

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0
    • Component/s: Client, regionserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HBASE-1845 introduced MultiGet and other cross-row/cross-region batch operations. We should add a way to do that with increments.

      1. 2947-final.txt
        0.9 kB
        Lars Hofhansl
      2. 2947-v2.txt
        5 kB
        Lars Hofhansl
      3. HBASE-2947-v1.patch
        2 kB
        Jonathan Gray

        Issue Links

          Activity

          Jonathan Gray created issue -
          Jonathan Gray made changes -
          Field Original Value New Value
          Link This issue is related to HBASE-2946 [ HBASE-2946 ]
          Hide
          Jonathan Gray added a comment -

          Most of what needs to be done is (trivially) make Increment implement Row.

          Show
          Jonathan Gray added a comment - Most of what needs to be done is (trivially) make Increment implement Row.
          Jonathan Gray made changes -
          Assignee Jonathan Gray [ streamy ]
          Hide
          Jonathan Gray added a comment -

          Changes Increment to implement Row and adds the check for instanceof Increment in HRS.multi

          Show
          Jonathan Gray added a comment - Changes Increment to implement Row and adds the check for instanceof Increment in HRS.multi
          Jonathan Gray made changes -
          Attachment HBASE-2947-v1.patch [ 12466668 ]
          Hide
          Jonathan Gray added a comment -

          Will work on a unit test and javadoc additions/cleanup.

          Show
          Jonathan Gray added a comment - Will work on a unit test and javadoc additions/cleanup.
          Hide
          Andrew Purtell added a comment -

          (Partial) dup of HBASE-2814?

          Show
          Andrew Purtell added a comment - (Partial) dup of HBASE-2814 ?
          Hide
          Jonathan Gray added a comment -

          HBASE-2814 seems to only be about thrift. This is to make Increment a Row operation so it can be used with the existing MultiAction stuff.

          Show
          Jonathan Gray added a comment - HBASE-2814 seems to only be about thrift. This is to make Increment a Row operation so it can be used with the existing MultiAction stuff.
          Hide
          Lars Hofhansl added a comment -

          @Jon: Are you still working on this? Patch looks good to me and we have recently found some need for this.
          I'm happy to commit, unless there's opposition.

          Show
          Lars Hofhansl added a comment - @Jon: Are you still working on this? Patch looks good to me and we have recently found some need for this. I'm happy to commit, unless there's opposition.
          Hide
          Jonathan Gray added a comment -

          Not working on it but no reason not to commit that I recall.

          Show
          Jonathan Gray added a comment - Not working on it but no reason not to commit that I recall.
          Hide
          Lars Hofhansl added a comment -

          Cool... I'll add a test.

          Show
          Lars Hofhansl added a comment - Cool... I'll add a test.
          Lars Hofhansl made changes -
          Assignee Jonathan Gray [ streamy ] Lars Hofhansl [ lhofhansl ]
          Hide
          Lars Hofhansl added a comment -

          While I am at it, I am also going to this for Append.

          Show
          Lars Hofhansl added a comment - While I am at it, I am also going to this for Append.
          Lars Hofhansl made changes -
          Summary MultiIncrement (MultiGet functionality for increments) MultiIncrement/MultiAppend (MultiGet functionality for increments and appends)
          Fix Version/s 0.94.0 [ 12316419 ]
          Hide
          Lars Hofhansl added a comment -

          Patch for trunk, includes changes for Append and a test.

          Show
          Lars Hofhansl added a comment - Patch for trunk, includes changes for Append and a test.
          Lars Hofhansl made changes -
          Attachment 2947-v2.txt [ 12509382 ]
          Lars Hofhansl made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12509382/2947-v2.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -151 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestLogRolling
          org.apache.hadoop.hbase.mapreduce.TestImportTsv
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/663//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/663//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/663//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12509382/2947-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -151 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 79 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/663//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/663//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/663//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment - - edited

          I ran the four failing tests locally and they all pass.
          I would like to commit this today (the change itself is pretty uncontentious I think).

          Show
          Lars Hofhansl added a comment - - edited I ran the four failing tests locally and they all pass. I would like to commit this today (the change itself is pretty uncontentious I think).
          Hide
          Lars Hofhansl added a comment -

          Any objections?

          Show
          Lars Hofhansl added a comment - Any objections?
          Hide
          stack added a comment -

          Is this safe to do Lars?

          -public class Increment implements Writable {
          +public class Increment implements Row {
          

          Should you leave Writable in place?

          Else LGTM

          Show
          stack added a comment - Is this safe to do Lars? - public class Increment implements Writable { + public class Increment implements Row { Should you leave Writable in place? Else LGTM
          Hide
          Lars Hofhansl added a comment -

          Good point... I just checked, and Row extends WritableComparable, which in turn extends Writable... So should be good.
          Maybe than Append can be simplified to only implement Row and not also Writable.

          Show
          Lars Hofhansl added a comment - Good point... I just checked, and Row extends WritableComparable, which in turn extends Writable... So should be good. Maybe than Append can be simplified to only implement Row and not also Writable.
          Hide
          Lars Hofhansl added a comment -

          Patch that I committed (based on Stack's comment Append does not actually need to implement Writable, since Row already extends Writable).

          Show
          Lars Hofhansl added a comment - Patch that I committed (based on Stack's comment Append does not actually need to implement Writable, since Row already extends Writable).
          Lars Hofhansl made changes -
          Attachment 2947-final.txt [ 12509472 ]
          Hide
          Lars Hofhansl added a comment -

          Thanks for the review Stack!

          Show
          Lars Hofhansl added a comment - Thanks for the review Stack!
          Lars Hofhansl made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2612 (See https://builds.apache.org/job/HBase-TRUNK/2612/)
          HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2612 (See https://builds.apache.org/job/HBase-TRUNK/2612/ ) HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H) larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #65 (See https://builds.apache.org/job/HBase-TRUNK-security/65/)
          HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H)

          larsh :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #65 (See https://builds.apache.org/job/HBase-TRUNK-security/65/ ) HBASE-2947 MultiIncrement/MultiAppend (JGray and Lars H) larsh : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Append.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Increment.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          490d 4h 32m 1 Lars Hofhansl 04/Jan/12 06:59
          Patch Available Patch Available Resolved Resolved
          16h 19m 1 Lars Hofhansl 04/Jan/12 23:19
          Resolved Resolved Closed Closed
          281d 6h 16m 1 Lars Hofhansl 12/Oct/12 06:35

            People

            • Assignee:
              Lars Hofhansl
              Reporter:
              Jonathan Gray
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development