Details

    • Hadoop Flags:
      Reviewed
    1. hbase-5620-sec.patch
      18 kB
      Jimmy Xiang
    2. hbase-5620_v4.patch
      3.23 MB
      Jimmy Xiang
    3. hbase-5620_v4.patch
      3.23 MB
      stack
    4. hbase-5620_v3.patch
      3.23 MB
      Jimmy Xiang

      Issue Links

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/
        -----------------------------------------------------------

        Review request for hbase.

        Summary
        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.
        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs


        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba
        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9
        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa
        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017
        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086
        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969
        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295
        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3
        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad
        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab
        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489
        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393
        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717
        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0
        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457
        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c
        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de
        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d
        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f
        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION
        src/main/protobuf/RegionClient.proto 358382b
        src/main/protobuf/hbase.proto da78788
        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc
        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468
        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc
        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3
        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060
        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36
        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2
        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5
        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989
        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing
        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6751
        -----------------------------------------------------------

        Good stuff Jimmy. A few comments below. Sorry I ramble at times. Read to the end before reacting because I change my mind as I work through the patch. You forgot to add RequestConverter?

        src/main/java/org/apache/hadoop/hbase/HConstants.java
        <https://reviews.apache.org/r/4629/#comment14787>

        Should these be in here? Can they be elsewhere, as defines in RegionClassProtocol?

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
        <https://reviews.apache.org/r/4629/#comment14788>

        I don't know how you get away with these removes but I'm glad to see them go – anything that shrinks this catalog package is good by me

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment14789>

        RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol?

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment14790>

        this should be getClient? We're going to get a client for a table? Maybe should be getTableClient?

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment14791>

        Why we need a converter?

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment14792>

        Whats the NULL_CONTROLLER about?

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment14793>

        So we don't close scanners anymore?

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java
        <https://reviews.apache.org/r/4629/#comment14794>

        Why do we need to pollute HConnection w/ higher level notions such as RegionClientProtocol (or ClientProtocol?).... hmmm ClientProtocol would be ok but not RegionClientProtocol...

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java
        <https://reviews.apache.org/r/4629/#comment14795>

        I wonder if this needs to be public and in the Interface? Could it be an internal thing? How uses this thing that comes out? Maybe it needs to be here but sure seems like an internal thing: i.e. we pass hostname and port of a particular regionserver but then internally as we go about the cluster the client will go to new regionservers and set up connections....

        Looking around though, it seems that while most uses of this method should be shut down: e.g. over in catalog package and replaced by calls that go via HTable, there are a few places we need this still probably .. as in the master sendRegionClose, etc. calls. So you can't get rid of it.

        But this looks like it should be called RegionServerProtocol
        or ClientProtocol and not RegionClientProtocol. It connects to a RegionServer not a Region.....

        Or what you think Jimmy? When I look at the RegionClient proto file, is it true to say that in each message, we MUST pass a region since we are going against a particular region... which would mean this is indeed a RegionClientProtocol (you just have to specify coordinates in two steps; 1. pass servername setting up connection, then 2. for any call on the connection, need to specify the region.....)

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment14796>

        Move these defines into this class rather than up in HConstants?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment14797>

        Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment14798>

        Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment14799>

        Does this need to be synchronized at all?

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        <https://reviews.apache.org/r/4629/#comment14800>

        I suppose HBaseClient down in rpc knows it because it has to deserialize the pb?

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        <https://reviews.apache.org/r/4629/#comment14801>

        scannerid is internal or something now?

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        <https://reviews.apache.org/r/4629/#comment14802>

        Not your fault but server is a bad variable name for what this is, huh..

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java
        <https://reviews.apache.org/r/4629/#comment14803>

        Do you have to? This stuff is nasty enough w/o lettting it out of this package.

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java
        <https://reviews.apache.org/r/4629/#comment14804>

        Fix

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java
        <https://reviews.apache.org/r/4629/#comment14805>

        What is this for?

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
        <https://reviews.apache.org/r/4629/#comment14806>

        Is ServiceException a pb exception?

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
        <https://reviews.apache.org/r/4629/#comment14807>

        Why not throw ServiceException?

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
        <https://reviews.apache.org/r/4629/#comment14808>

        Ugh. KeyValue needs to be an Interface that both pb and our current KV implements.

        This will be a performance killer? Should Result let out the raw pb KV?

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
        <https://reviews.apache.org/r/4629/#comment14809>

        Good one. I should use this in stuff I committed recently

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
        <https://reviews.apache.org/r/4629/#comment14810>

        Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc.

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java
        <https://reviews.apache.org/r/4629/#comment14811>

        white space

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java
        <https://reviews.apache.org/r/4629/#comment14812>

        Is this just regionadmin protocol? Or is it general admin?

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java
        <https://reviews.apache.org/r/4629/#comment14813>

        This is probably a good idea.

        • Michael

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6751 ----------------------------------------------------------- Good stuff Jimmy. A few comments below. Sorry I ramble at times. Read to the end before reacting because I change my mind as I work through the patch. You forgot to add RequestConverter? src/main/java/org/apache/hadoop/hbase/HConstants.java < https://reviews.apache.org/r/4629/#comment14787 > Should these be in here? Can they be elsewhere, as defines in RegionClassProtocol? src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java < https://reviews.apache.org/r/4629/#comment14788 > I don't know how you get away with these removes but I'm glad to see them go – anything that shrinks this catalog package is good by me src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment14789 > RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol? src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment14790 > this should be getClient? We're going to get a client for a table? Maybe should be getTableClient? src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment14791 > Why we need a converter? src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment14792 > Whats the NULL_CONTROLLER about? src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment14793 > So we don't close scanners anymore? src/main/java/org/apache/hadoop/hbase/client/HConnection.java < https://reviews.apache.org/r/4629/#comment14794 > Why do we need to pollute HConnection w/ higher level notions such as RegionClientProtocol (or ClientProtocol?).... hmmm ClientProtocol would be ok but not RegionClientProtocol... src/main/java/org/apache/hadoop/hbase/client/HConnection.java < https://reviews.apache.org/r/4629/#comment14795 > I wonder if this needs to be public and in the Interface? Could it be an internal thing? How uses this thing that comes out? Maybe it needs to be here but sure seems like an internal thing: i.e. we pass hostname and port of a particular regionserver but then internally as we go about the cluster the client will go to new regionservers and set up connections.... Looking around though, it seems that while most uses of this method should be shut down: e.g. over in catalog package and replaced by calls that go via HTable, there are a few places we need this still probably .. as in the master sendRegionClose, etc. calls. So you can't get rid of it. But this looks like it should be called RegionServerProtocol or ClientProtocol and not RegionClientProtocol. It connects to a RegionServer not a Region..... Or what you think Jimmy? When I look at the RegionClient proto file, is it true to say that in each message, we MUST pass a region since we are going against a particular region... which would mean this is indeed a RegionClientProtocol (you just have to specify coordinates in two steps; 1. pass servername setting up connection, then 2. for any call on the connection, need to specify the region.....) src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment14796 > Move these defines into this class rather than up in HConstants? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment14797 > Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment14798 > Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment14799 > Does this need to be synchronized at all? src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java < https://reviews.apache.org/r/4629/#comment14800 > I suppose HBaseClient down in rpc knows it because it has to deserialize the pb? src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java < https://reviews.apache.org/r/4629/#comment14801 > scannerid is internal or something now? src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java < https://reviews.apache.org/r/4629/#comment14802 > Not your fault but server is a bad variable name for what this is, huh.. src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java < https://reviews.apache.org/r/4629/#comment14803 > Do you have to? This stuff is nasty enough w/o lettting it out of this package. src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java < https://reviews.apache.org/r/4629/#comment14804 > Fix src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java < https://reviews.apache.org/r/4629/#comment14805 > What is this for? src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java < https://reviews.apache.org/r/4629/#comment14806 > Is ServiceException a pb exception? src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java < https://reviews.apache.org/r/4629/#comment14807 > Why not throw ServiceException? src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java < https://reviews.apache.org/r/4629/#comment14808 > Ugh. KeyValue needs to be an Interface that both pb and our current KV implements. This will be a performance killer? Should Result let out the raw pb KV? src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java < https://reviews.apache.org/r/4629/#comment14809 > Good one. I should use this in stuff I committed recently src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java < https://reviews.apache.org/r/4629/#comment14810 > Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc. src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java < https://reviews.apache.org/r/4629/#comment14811 > white space src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java < https://reviews.apache.org/r/4629/#comment14812 > Is this just regionadmin protocol? Or is it general admin? src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java < https://reviews.apache.org/r/4629/#comment14813 > This is probably a good idea. Michael On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > Good stuff Jimmy. A few comments below. Sorry I ramble at times. Read to the end before reacting because I change my mind as I work through the patch. You forgot to add RequestConverter?

        Thanks for reviewing. RequestConverter is the 21st file. There are more files on page 2. Please review them too. Thanks.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 766

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line766>

        >

        > white space

        Will remove.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java, line 37

        > <https://reviews.apache.org/r/4629/diff/1/?file=98833#file98833line37>

        >

        > Is this just regionadmin protocol? Or is it general admin?

        Should I change it to AdminProtocol? There is no other admin.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 510

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line510>

        >

        > Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc.

        I agree. We can add some new methods later. For now, I'd like to make the existing functions work at first.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 160

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line160>

        >

        > Ugh. KeyValue needs to be an Interface that both pb and our current KV implements.

        >

        > This will be a performance killer? Should Result let out the raw pb KV?

        I was thinking to has some new methods which use raw pbs, which old clients should migrate to.

        Let me change the current KeyValue class to an interface.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 149

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line149>

        >

        > Why not throw ServiceException?

        I like to throw ServiceException too. Let me give it a try. If the existing interface is not affected, I will do it.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java, line 418

        > <https://reviews.apache.org/r/4629/diff/1/?file=98830#file98830line418>

        >

        > Is ServiceException a pb exception?

        It is from pb. But it is not a pb message as I know.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36

        > <https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36>

        >

        > What is this for?

        Region server could implement multiple protocols such as RegionClientProtocol and RegionAdminProtocol.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 27

        > <https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line27>

        >

        > Fix

        Fixed.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 363

        > <https://reviews.apache.org/r/4629/diff/1/?file=98825#file98825line363>

        >

        > Do you have to? This stuff is nasty enough w/o lettting it out of this package.

        It is to support Filters and comparators which are still Writable. We can change it back once Filters and comparators are moved to pb too.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 230

        > <https://reviews.apache.org/r/4629/diff/1/?file=98822#file98822line230>

        >

        > scannerid is internal or something now?

        I think it is still public as it is defined in the pb messages. For ScannerCallable, it is internal though.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 192

        > <https://reviews.apache.org/r/4629/diff/1/?file=98822#file98822line192>

        >

        > I suppose HBaseClient down in rpc knows it because it has to deserialize the pb?

        You are right. We need to come up a way to pop this info up.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1441

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1441>

        >

        > Does this need to be synchronized at all?

        I think so. It's better to be synchronized. The connection is shared.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1428

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1428>

        >

        > Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one?

        In this case, they need to close the existing connection, and get a new one. Otherwise, there will be memory leak.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 606

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line606>

        >

        > Move these defines into this class rather than up in HConstants?

        Ok, will do.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1395

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1395>

        >

        > Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername?

        Just did some refactory so that we can use the same method to get a proxy for either a RegionClientProtocol and a HRegionInterface.

        The public interface is not touched.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 43

        > <https://reviews.apache.org/r/4629/diff/1/?file=98819#file98819line43>

        >

        > Why do we need to pollute HConnection w/ higher level notions such as RegionClientProtocol (or ClientProtocol?).... hmmm ClientProtocol would be ok but not RegionClientProtocol...

        OK, will rename it to ClientProtocol.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 530

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line530>

        >

        > Whats the NULL_CONTROLLER about?

        Pb rpc needs a RpcController. Let me use null directly.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line574>

        >

        > So we don't close scanners anymore?

        That's right, if closeScanner is specified in the request.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 525

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line525>

        >

        > Why we need a converter?

        Pb needs some work to build a message. A converter is some helper to share some logic. The ProtobufUtil will be too big soon. That's why I have this separate converter.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 518

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line518>

        >

        > this should be getClient? We're going to get a client for a table? Maybe should be getTableClient?

        The region client protocol. How about just getClient?

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 517

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line517>

        >

        > RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol?

        Ok, will change to ClientProtocol and AdminProtocol.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/HConstants.java, line 661

        > <https://reviews.apache.org/r/4629/diff/1/?file=98815#file98815line661>

        >

        > Should these be in here? Can they be elsewhere, as defines in RegionClassProtocol?

        Will move to HConnectionManager.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 234

        > <https://reviews.apache.org/r/4629/diff/1/?file=98819#file98819line234>

        >

        > I wonder if this needs to be public and in the Interface? Could it be an internal thing? How uses this thing that comes out? Maybe it needs to be here but sure seems like an internal thing: i.e. we pass hostname and port of a particular regionserver but then internally as we go about the cluster the client will go to new regionservers and set up connections....

        >

        > Looking around though, it seems that while most uses of this method should be shut down: e.g. over in catalog package and replaced by calls that go via HTable, there are a few places we need this still probably .. as in the master sendRegionClose, etc. calls. So you can't get rid of it.

        >

        > But this looks like it should be called RegionServerProtocol

        > or ClientProtocol and not RegionClientProtocol. It connects to a RegionServer not a Region.....

        >

        > Or what you think Jimmy? When I look at the RegionClient proto file, is it true to say that in each message, we MUST pass a region since we are going against a particular region... which would mean this is indeed a RegionClientProtocol (you just have to specify coordinates in two steps; 1. pass servername setting up connection, then 2. for any call on the connection, need to specify the region.....)

        >

        >

        >

        Agreed to rename it to ClientProtocol.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6751
        -----------------------------------------------------------

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-06 21:02:20, Michael Stack wrote: > Good stuff Jimmy. A few comments below. Sorry I ramble at times. Read to the end before reacting because I change my mind as I work through the patch. You forgot to add RequestConverter? Thanks for reviewing. RequestConverter is the 21st file. There are more files on page 2. Please review them too. Thanks. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 766 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line766 > > > white space Will remove. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java, line 37 > < https://reviews.apache.org/r/4629/diff/1/?file=98833#file98833line37 > > > Is this just regionadmin protocol? Or is it general admin? Should I change it to AdminProtocol? There is no other admin. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 510 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line510 > > > Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc. I agree. We can add some new methods later. For now, I'd like to make the existing functions work at first. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 160 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line160 > > > Ugh. KeyValue needs to be an Interface that both pb and our current KV implements. > > This will be a performance killer? Should Result let out the raw pb KV? I was thinking to has some new methods which use raw pbs, which old clients should migrate to. Let me change the current KeyValue class to an interface. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 149 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line149 > > > Why not throw ServiceException? I like to throw ServiceException too. Let me give it a try. If the existing interface is not affected, I will do it. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java, line 418 > < https://reviews.apache.org/r/4629/diff/1/?file=98830#file98830line418 > > > Is ServiceException a pb exception? It is from pb. But it is not a pb message as I know. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36 > < https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36 > > > What is this for? Region server could implement multiple protocols such as RegionClientProtocol and RegionAdminProtocol. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 27 > < https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line27 > > > Fix Fixed. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 363 > < https://reviews.apache.org/r/4629/diff/1/?file=98825#file98825line363 > > > Do you have to? This stuff is nasty enough w/o lettting it out of this package. It is to support Filters and comparators which are still Writable. We can change it back once Filters and comparators are moved to pb too. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 230 > < https://reviews.apache.org/r/4629/diff/1/?file=98822#file98822line230 > > > scannerid is internal or something now? I think it is still public as it is defined in the pb messages. For ScannerCallable, it is internal though. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 192 > < https://reviews.apache.org/r/4629/diff/1/?file=98822#file98822line192 > > > I suppose HBaseClient down in rpc knows it because it has to deserialize the pb? You are right. We need to come up a way to pop this info up. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1441 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1441 > > > Does this need to be synchronized at all? I think so. It's better to be synchronized. The connection is shared. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1428 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1428 > > > Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one? In this case, they need to close the existing connection, and get a new one. Otherwise, there will be memory leak. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 606 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line606 > > > Move these defines into this class rather than up in HConstants? Ok, will do. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1395 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1395 > > > Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername? Just did some refactory so that we can use the same method to get a proxy for either a RegionClientProtocol and a HRegionInterface. The public interface is not touched. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 43 > < https://reviews.apache.org/r/4629/diff/1/?file=98819#file98819line43 > > > Why do we need to pollute HConnection w/ higher level notions such as RegionClientProtocol (or ClientProtocol?).... hmmm ClientProtocol would be ok but not RegionClientProtocol... OK, will rename it to ClientProtocol. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 530 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line530 > > > Whats the NULL_CONTROLLER about? Pb rpc needs a RpcController. Let me use null directly. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line574 > > > So we don't close scanners anymore? That's right, if closeScanner is specified in the request. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 525 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line525 > > > Why we need a converter? Pb needs some work to build a message. A converter is some helper to share some logic. The ProtobufUtil will be too big soon. That's why I have this separate converter. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 518 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line518 > > > this should be getClient? We're going to get a client for a table? Maybe should be getTableClient? The region client protocol. How about just getClient? On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 517 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line517 > > > RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol? Ok, will change to ClientProtocol and AdminProtocol. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/HConstants.java, line 661 > < https://reviews.apache.org/r/4629/diff/1/?file=98815#file98815line661 > > > Should these be in here? Can they be elsewhere, as defines in RegionClassProtocol? Will move to HConnectionManager. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 234 > < https://reviews.apache.org/r/4629/diff/1/?file=98819#file98819line234 > > > I wonder if this needs to be public and in the Interface? Could it be an internal thing? How uses this thing that comes out? Maybe it needs to be here but sure seems like an internal thing: i.e. we pass hostname and port of a particular regionserver but then internally as we go about the cluster the client will go to new regionservers and set up connections.... > > Looking around though, it seems that while most uses of this method should be shut down: e.g. over in catalog package and replaced by calls that go via HTable, there are a few places we need this still probably .. as in the master sendRegionClose, etc. calls. So you can't get rid of it. > > But this looks like it should be called RegionServerProtocol > or ClientProtocol and not RegionClientProtocol. It connects to a RegionServer not a Region..... > > Or what you think Jimmy? When I look at the RegionClient proto file, is it true to say that in each message, we MUST pass a region since we are going against a particular region... which would mean this is indeed a RegionClientProtocol (you just have to specify coordinates in two steps; 1. pass servername setting up connection, then 2. for any call on the connection, need to specify the region.....) > > > Agreed to rename it to ClientProtocol. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6751 ----------------------------------------------------------- On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6757
        -----------------------------------------------------------

        I missed this second page. Looks great. Some radical stuff going on in here. Can you explain some? Good on you Jimmy.

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14844>

        What does this do? Needs class comment

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14845>

        Spelling

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14846>

        We only do the latter if the flag is set, right?

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14847>

        Is this a get request builder? Name method getRequestBuilder?

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14848>

        Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now?

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14849>

        buildMutateRequest?

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java
        <https://reviews.apache.org/r/4629/#comment14850>

        Should this stuff be public? Its internal stuff? Should it be over under the client package? Then you could keep it encapsulated at least w/i the package? Is it used outside of the client package?

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        <https://reviews.apache.org/r/4629/#comment14851>

        Ditto. Should this be over in the client package? Is that the only place its used? Could we make it non-public?

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        <https://reviews.apache.org/r/4629/#comment14852>

        buildActionResult?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14864>

        We don't need this anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14863>

        Why we don't need this anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14862>

        We don't need this anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14861>

        We don't need any more?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14860>

        We don't need this anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14859>

        For sure these are no longer needed?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14858>

        Not needed anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14857>

        Not used anymore?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14856>

        Why this no more?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14855>

        Why this no more?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14854>

        No more of this?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14865>

        Ain't this needed?

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        <https://reviews.apache.org/r/4629/#comment14866>

        Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go)

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
        <https://reviews.apache.org/r/4629/#comment14867>

        oh... I think I like what I'm seeing. Could this be an Interface at all?

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
        <https://reviews.apache.org/r/4629/#comment14868>

        What is the advantage of doing this?

        src/main/protobuf/RegionClient.proto
        <https://reviews.apache.org/r/4629/#comment14869>

        We don't need these?

        src/main/protobuf/hbase.proto
        <https://reviews.apache.org/r/4629/#comment14870>

        Good

        • Michael

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6757 ----------------------------------------------------------- I missed this second page. Looks great. Some radical stuff going on in here. Can you explain some? Good on you Jimmy. src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14844 > What does this do? Needs class comment src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14845 > Spelling src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14846 > We only do the latter if the flag is set, right? src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14847 > Is this a get request builder? Name method getRequestBuilder? src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14848 > Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now? src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14849 > buildMutateRequest? src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java < https://reviews.apache.org/r/4629/#comment14850 > Should this stuff be public? Its internal stuff? Should it be over under the client package? Then you could keep it encapsulated at least w/i the package? Is it used outside of the client package? src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java < https://reviews.apache.org/r/4629/#comment14851 > Ditto. Should this be over in the client package? Is that the only place its used? Could we make it non-public? src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java < https://reviews.apache.org/r/4629/#comment14852 > buildActionResult? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14864 > We don't need this anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14863 > Why we don't need this anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14862 > We don't need this anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14861 > We don't need any more? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14860 > We don't need this anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14859 > For sure these are no longer needed? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14858 > Not needed anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14857 > Not used anymore? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14856 > Why this no more? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14855 > Why this no more? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14854 > No more of this? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14865 > Ain't this needed? src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java < https://reviews.apache.org/r/4629/#comment14866 > Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go) src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java < https://reviews.apache.org/r/4629/#comment14867 > oh... I think I like what I'm seeing. Could this be an Interface at all? src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java < https://reviews.apache.org/r/4629/#comment14868 > What is the advantage of doing this? src/main/protobuf/RegionClient.proto < https://reviews.apache.org/r/4629/#comment14869 > We don't need these? src/main/protobuf/hbase.proto < https://reviews.apache.org/r/4629/#comment14870 > Good Michael On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6759
        -----------------------------------------------------------

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java
        <https://reviews.apache.org/r/4629/#comment14879>

        Where is RegionProtocols used? We put the two protocols together? Do we need to?

        • Michael

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6759 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java < https://reviews.apache.org/r/4629/#comment14879 > Where is RegionProtocols used? We put the two protocols together? Do we need to? Michael On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-06 23:41:33, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36

        > <https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36>

        >

        > Where is RegionProtocols used? We put the two protocols together? Do we need to?

        Ok, let me remove it.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6759
        -----------------------------------------------------------

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-06 23:41:33, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36 > < https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36 > > > Where is RegionProtocols used? We put the two protocols together? Do we need to? Ok, let me remove it. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6759 ----------------------------------------------------------- On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 89

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line89>

        >

        > What does this do? Needs class comment

        Will add comment.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 97

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line97>

        >

        > Spelling

        Good catch, will fix.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 98

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line98>

        >

        > We only do the latter if the flag is set, right?

        Right.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 107

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line107>

        >

        > Is this a get request builder? Name method getRequestBuilder?

        It uses a request builder to build a get request.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 146

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line146>

        >

        > Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now?

        I think so. We don't want to break the existing applications.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 167

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line167>

        >

        > buildMutateRequest?

        Sounds good to me.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 646

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line646>

        >

        > Should this stuff be public? Its internal stuff? Should it be over under the client package? Then you could keep it encapsulated at least w/i the package? Is it used outside of the client package?

        Let me check. I will try not to expose it.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 56

        > <https://reviews.apache.org/r/4629/diff/1/?file=98836#file98836line56>

        >

        > Ditto. Should this be over in the client package? Is that the only place its used? Could we make it non-public?

        Will do if it works.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 101

        > <https://reviews.apache.org/r/4629/diff/1/?file=98836#file98836line101>

        >

        > buildActionResult?

        Sure.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 185

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line185>

        >

        > We don't need this anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 193

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line193>

        >

        > Why we don't need this anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 218

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line218>

        >

        > We don't need this anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 229

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line229>

        >

        > We don't need any more?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 271

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line271>

        >

        > We don't need this anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 459

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line459>

        >

        > For sure these are no longer needed?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1119

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1119>

        >

        > Not needed anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1155

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1155>

        >

        > Not used anymore?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1181

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1181>

        >

        > Why this no more?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1199

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1199>

        >

        > Why this no more?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 2518

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line2518>

        >

        > No more of this?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3094

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3094>

        >

        > Ain't this needed?

        Moved to RegionServer, a new base class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3195

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3195>

        >

        > Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go)

        Moved to RegionServer, a new base class. I was think to remove class HRegionServer later on when HRegionInterface is not needed.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98

        > <https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line98>

        >

        > oh... I think I like what I'm seeing. Could this be an Interface at all?

        It is the new pb version of HRegionServer. So it is a class.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/protobuf/RegionClient.proto, line 150

        > <https://reviews.apache.org/r/4629/diff/1/?file=98843#file98843line150>

        >

        > We don't need these?

        I added DeleteType to handle such usecases.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 1162

        > <https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line1162>

        >

        > What is the advantage of doing this?

        So I don't mess HRegionInterface with the new pb. Later on, it will be easier to remove HRegionInterface.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6757
        -----------------------------------------------------------

        On 2012-04-03 23:32:10, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-03 23:32:10)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969

        src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 89 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line89 > > > What does this do? Needs class comment Will add comment. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 97 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line97 > > > Spelling Good catch, will fix. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 98 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line98 > > > We only do the latter if the flag is set, right? Right. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 107 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line107 > > > Is this a get request builder? Name method getRequestBuilder? It uses a request builder to build a get request. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 146 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line146 > > > Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now? I think so. We don't want to break the existing applications. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 167 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line167 > > > buildMutateRequest? Sounds good to me. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 646 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line646 > > > Should this stuff be public? Its internal stuff? Should it be over under the client package? Then you could keep it encapsulated at least w/i the package? Is it used outside of the client package? Let me check. I will try not to expose it. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 56 > < https://reviews.apache.org/r/4629/diff/1/?file=98836#file98836line56 > > > Ditto. Should this be over in the client package? Is that the only place its used? Could we make it non-public? Will do if it works. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 101 > < https://reviews.apache.org/r/4629/diff/1/?file=98836#file98836line101 > > > buildActionResult? Sure. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 185 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line185 > > > We don't need this anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 193 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line193 > > > Why we don't need this anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 218 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line218 > > > We don't need this anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 229 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line229 > > > We don't need any more? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 271 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line271 > > > We don't need this anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 459 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line459 > > > For sure these are no longer needed? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1119 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1119 > > > Not needed anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1155 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1155 > > > Not used anymore? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1181 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1181 > > > Why this no more? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 1199 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line1199 > > > Why this no more? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 2518 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line2518 > > > No more of this? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3094 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3094 > > > Ain't this needed? Moved to RegionServer, a new base class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3195 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3195 > > > Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go) Moved to RegionServer, a new base class. I was think to remove class HRegionServer later on when HRegionInterface is not needed. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98 > < https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line98 > > > oh... I think I like what I'm seeing. Could this be an Interface at all? It is the new pb version of HRegionServer. So it is a class. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/protobuf/RegionClient.proto, line 150 > < https://reviews.apache.org/r/4629/diff/1/?file=98843#file98843line150 > > > We don't need these? I added DeleteType to handle such usecases. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 1162 > < https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line1162 > > > What is the advantage of doing this? So I don't mess HRegionInterface with the new pb. Later on, it will be easier to remove HRegionInterface. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6757 ----------------------------------------------------------- On 2012-04-03 23:32:10, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-03 23:32:10) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/HConstants.java 21ac4ba src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java aa30969 src/main/java/org/apache/hadoop/hbase/client/HTable.java 3db0295 src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RegionClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 4f80999 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java d2b3060 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 91dce36 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 227c5f2 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/
        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32.906966)

        Review request for hbase.

        Changes
        -------

        Addressed Stack's comments. For KeyValue, I removed it from the pb Result.

        Summary
        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.
        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs (updated)


        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9
        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa
        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017
        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086
        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce
        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f
        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3
        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad
        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab
        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489
        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393
        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717
        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0
        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457
        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c
        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de
        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d
        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f
        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION
        src/main/protobuf/Admin.proto PRE-CREATION
        src/main/protobuf/Client.proto PRE-CREATION
        src/main/protobuf/RegionAdmin.proto c64d68b
        src/main/protobuf/RegionClient.proto 358382b
        src/main/protobuf/hbase.proto da78788
        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc
        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468
        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc
        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3
        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8
        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2
        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83
        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5
        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989
        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing
        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32.906966) Review request for hbase. Changes ------- Addressed Stack's comments. For KeyValue, I removed it from the pb Result. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs (updated) src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 517

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line517>

        >

        > RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol?

        Jimmy Xiang wrote:

        Ok, will change to ClientProtocol and AdminProtocol.

        I think thats better.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 518

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line518>

        >

        > this should be getClient? We're going to get a client for a table? Maybe should be getTableClient?

        Jimmy Xiang wrote:

        The region client protocol. How about just getClient?

        Yes. In actuality, isn't this really a client on a region so getTableClient wouldn't be correct either as I suggest above.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 525

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line525>

        >

        > Why we need a converter?

        Jimmy Xiang wrote:

        Pb needs some work to build a message. A converter is some helper to share some logic. The ProtobufUtil will be too big soon. That's why I have this separate converter.

        Good. I saw that later on.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 530

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line530>

        >

        > Whats the NULL_CONTROLLER about?

        Jimmy Xiang wrote:

        Pb rpc needs a RpcController. Let me use null directly.

        yeah...

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574

        > <https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line574>

        >

        > So we don't close scanners anymore?

        Jimmy Xiang wrote:

        That's right, if closeScanner is specified in the request.

        oh, nice.

        What about the case where no close is called. We'll hold on to the resources on the server side w/o cleanup?

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1395

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1395>

        >

        > Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername?

        Jimmy Xiang wrote:

        Just did some refactory so that we can use the same method to get a proxy for either a RegionClientProtocol and a HRegionInterface.

        The public interface is not touched.

        ok. good.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1428

        > <https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1428>

        >

        > Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one?

        Jimmy Xiang wrote:

        In this case, they need to close the existing connection, and get a new one. Otherwise, there will be memory leak.

        Otherwise, we will overwrite the old with the new?

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 363

        > <https://reviews.apache.org/r/4629/diff/1/?file=98825#file98825line363>

        >

        > Do you have to? This stuff is nasty enough w/o lettting it out of this package.

        Jimmy Xiang wrote:

        It is to support Filters and comparators which are still Writable. We can change it back once Filters and comparators are moved to pb too.

        ok

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36

        > <https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36>

        >

        > What is this for?

        Jimmy Xiang wrote:

        Region server could implement multiple protocols such as RegionClientProtocol and RegionAdminProtocol.

        Do we need to put them together? Who uses this RegionProtocols?

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 160

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line160>

        >

        > Ugh. KeyValue needs to be an Interface that both pb and our current KV implements.

        >

        > This will be a performance killer? Should Result let out the raw pb KV?

        Jimmy Xiang wrote:

        I was thinking to has some new methods which use raw pbs, which old clients should migrate to.

        Let me change the current KeyValue class to an interface.

        That might be hard boss. But, could we implement a subset of KV as an Interface, enough of a subset that it'd make these pb stuff happy?

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 510

        > <https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line510>

        >

        > Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc.

        Jimmy Xiang wrote:

        I agree. We can add some new methods later. For now, I'd like to make the existing functions work at first.

        Agreed. Looking at the later stuff I think Get, Delete, etc. should just go and we should have folks use the pb stuff.

        On 2012-04-06 21:02:20, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java, line 37

        > <https://reviews.apache.org/r/4629/diff/1/?file=98833#file98833line37>

        >

        > Is this just regionadmin protocol? Or is it general admin?

        Jimmy Xiang wrote:

        Should I change it to AdminProtocol? There is no other admin.

        Yes

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6751
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 517 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line517 > > > RegionClientProtocol is a bad name, no? Should it be just ClientProtocol and AdminProtocol? Jimmy Xiang wrote: Ok, will change to ClientProtocol and AdminProtocol. I think thats better. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 518 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line518 > > > this should be getClient? We're going to get a client for a table? Maybe should be getTableClient? Jimmy Xiang wrote: The region client protocol. How about just getClient? Yes. In actuality, isn't this really a client on a region so getTableClient wouldn't be correct either as I suggest above. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 525 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line525 > > > Why we need a converter? Jimmy Xiang wrote: Pb needs some work to build a message. A converter is some helper to share some logic. The ProtobufUtil will be too big soon. That's why I have this separate converter. Good. I saw that later on. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 530 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line530 > > > Whats the NULL_CONTROLLER about? Jimmy Xiang wrote: Pb rpc needs a RpcController. Let me use null directly. yeah... On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574 > < https://reviews.apache.org/r/4629/diff/1/?file=98818#file98818line574 > > > So we don't close scanners anymore? Jimmy Xiang wrote: That's right, if closeScanner is specified in the request. oh, nice. What about the case where no close is called. We'll hold on to the resources on the server side w/o cleanup? On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1395 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1395 > > > Whats happening here? We want to drop the methods that took hostname and port only and move to those that take servername? Jimmy Xiang wrote: Just did some refactory so that we can use the same method to get a proxy for either a RegionClientProtocol and a HRegionInterface. The public interface is not touched. ok. good. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1428 > < https://reviews.apache.org/r/4629/diff/1/?file=98820#file98820line1428 > > > Is this good? Should we have servername? What if server restarts in between? Would we want to notice that we don't have a connection to new instantiation and go create one? Jimmy Xiang wrote: In this case, they need to close the existing connection, and get a new one. Otherwise, there will be memory leak. Otherwise, we will overwrite the old with the new? On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java, line 363 > < https://reviews.apache.org/r/4629/diff/1/?file=98825#file98825line363 > > > Do you have to? This stuff is nasty enough w/o lettting it out of this package. Jimmy Xiang wrote: It is to support Filters and comparators which are still Writable. We can change it back once Filters and comparators are moved to pb too. ok On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/RegionProtocols.java, line 36 > < https://reviews.apache.org/r/4629/diff/1/?file=98829#file98829line36 > > > What is this for? Jimmy Xiang wrote: Region server could implement multiple protocols such as RegionClientProtocol and RegionAdminProtocol. Do we need to put them together? Who uses this RegionProtocols? On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 160 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line160 > > > Ugh. KeyValue needs to be an Interface that both pb and our current KV implements. > > This will be a performance killer? Should Result let out the raw pb KV? Jimmy Xiang wrote: I was thinking to has some new methods which use raw pbs, which old clients should migrate to. Let me change the current KeyValue class to an interface. That might be hard boss. But, could we implement a subset of KV as an Interface, enough of a subset that it'd make these pb stuff happy? On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java, line 510 > < https://reviews.apache.org/r/4629/diff/1/?file=98832#file98832line510 > > > Its as though we should give clients a way to pass in the raw pbs rather than have them go via our versions of Get, Delete, etc. Jimmy Xiang wrote: I agree. We can add some new methods later. For now, I'd like to make the existing functions work at first. Agreed. Looking at the later stuff I think Get, Delete, etc. should just go and we should have folks use the pb stuff. On 2012-04-06 21:02:20, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RegionAdminProtocol.java, line 37 > < https://reviews.apache.org/r/4629/diff/1/?file=98833#file98833line37 > > > Is this just regionadmin protocol? Or is it general admin? Jimmy Xiang wrote: Should I change it to AdminProtocol? There is no other admin. Yes Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6751 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 146

        > <https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line146>

        >

        > Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now?

        Jimmy Xiang wrote:

        I think so. We don't want to break the existing applications.

        No. As you suggest, parity first. Thinking on it, I'm thinking that when we write the new client, that it is the new client that just does pbs direct.

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3195

        > <https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3195>

        >

        > Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go)

        Jimmy Xiang wrote:

        Moved to RegionServer, a new base class. I was think to remove class HRegionServer later on when HRegionInterface is not needed.

        Ok. Would be really cool if there was a RegionServer Interface that the pb service implemented (We already have Server Interface and RegionServerServices Interface).

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98

        > <https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line98>

        >

        > oh... I think I like what I'm seeing. Could this be an Interface at all?

        Jimmy Xiang wrote:

        It is the new pb version of HRegionServer. So it is a class.

        Could it be an Interface that the pb Service implements? Is that over the top?

        On 2012-04-06 23:26:21, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 1162

        > <https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line1162>

        >

        > What is the advantage of doing this?

        Jimmy Xiang wrote:

        So I don't mess HRegionInterface with the new pb. Later on, it will be easier to remove HRegionInterface.

        I like this tactic of yours

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6757
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java, line 146 > < https://reviews.apache.org/r/4629/diff/1/?file=98835#file98835line146 > > > Do you think that we should eventually just deprecate the current Get, Delete, and Put, etc. and move to the pb ones altogether? They are only in the way now? Jimmy Xiang wrote: I think so. We don't want to break the existing applications. No. As you suggest, parity first. Thinking on it, I'm thinking that when we write the new client, that it is the new client that just does pbs direct. On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java, line 3195 > < https://reviews.apache.org/r/4629/diff/1/?file=98839#file98839line3195 > > > Some pretty radical removals in here. For sure we don't need the stuff being removed (Otherwise, I'm glad to see it go) Jimmy Xiang wrote: Moved to RegionServer, a new base class. I was think to remove class HRegionServer later on when HRegionInterface is not needed. Ok. Would be really cool if there was a RegionServer Interface that the pb service implemented (We already have Server Interface and RegionServerServices Interface). On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98 > < https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line98 > > > oh... I think I like what I'm seeing. Could this be an Interface at all? Jimmy Xiang wrote: It is the new pb version of HRegionServer. So it is a class. Could it be an Interface that the pb Service implements? Is that over the top? On 2012-04-06 23:26:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 1162 > < https://reviews.apache.org/r/4629/diff/1/?file=98842#file98842line1162 > > > What is the advantage of doing this? Jimmy Xiang wrote: So I don't mess HRegionInterface with the new pb. Later on, it will be easier to remove HRegionInterface. I like this tactic of yours Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6757 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6833
        -----------------------------------------------------------

        I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow.

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java
        <https://reviews.apache.org/r/4629/#comment15218>

        I said it before but I like this code removal

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment15219>

        What about the EOFE you added today? Does that need to go in here anywhere?

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
        <https://reviews.apache.org/r/4629/#comment15220>

        So, to be clear, if a client does not close a scanner now, how does it get cleaned up on the server? Does it live forever?

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java
        <https://reviews.apache.org/r/4629/#comment15221>

        Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think?

        Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this.

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15222>

        Rename to hbase.clientprotocol.class and CLIENT_PROTOCOL_CLASS. Seems more consistent?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15223>

        Rename too accordingly.

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15225>

        This is pb VersionedProtocol?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15224>

        Want to adjust this message?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15226>

        This set of protocols is covered by the synchronize held above? i.e. all access on protocols are under a block like this?

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java
        <https://reviews.apache.org/r/4629/#comment15227>

        Is there a difference here? We create an ISA only when we need it previously? Now we create it always? Is there a diff here?

        • Michael

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6833 ----------------------------------------------------------- I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow. src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java < https://reviews.apache.org/r/4629/#comment15218 > I said it before but I like this code removal src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment15219 > What about the EOFE you added today? Does that need to go in here anywhere? src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java < https://reviews.apache.org/r/4629/#comment15220 > So, to be clear, if a client does not close a scanner now, how does it get cleaned up on the server? Does it live forever? src/main/java/org/apache/hadoop/hbase/client/HConnection.java < https://reviews.apache.org/r/4629/#comment15221 > Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think? Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15222 > Rename to hbase.clientprotocol.class and CLIENT_PROTOCOL_CLASS. Seems more consistent? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15223 > Rename too accordingly. src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15225 > This is pb VersionedProtocol? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15224 > Want to adjust this message? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15226 > This set of protocols is covered by the synchronize held above? i.e. all access on protocols are under a block like this? src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java < https://reviews.apache.org/r/4629/#comment15227 > Is there a difference here? We create an ISA only when we need it previously? Now we create it always? Is there a diff here? Michael On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow.

        Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691

        > <https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691>

        >

        > I said it before but I like this code removal

        It is not used so I removed it.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 533

        > <https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line533>

        >

        > What about the EOFE you added today? Does that need to go in here anywhere?

        EOFE is an IOException. Good catch. I addressed it in HBASE-5621, which is in testing now.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574

        > <https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line574>

        >

        > So, to be clear, if a client does not close a scanner now, how does it get cleaned up on the server? Does it live forever?

        In this case, the Leases daemon will expire the scanner and clean it up.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 184

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line184>

        >

        > Rename to hbase.clientprotocol.class and CLIENT_PROTOCOL_CLASS. Seems more consistent?

        Will fix.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235

        > <https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235>

        >

        > Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think?

        >

        > Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this.

        I agree a ClientProtocol should have an HConnection. We can hide the HConnection. Given a region location, we can create a ClientProtocol which manages its own HConnection.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 187

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line187>

        >

        > Rename too accordingly.

        Will fix.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 565

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line565>

        >

        > This is pb VersionedProtocol?

        So far, it can be any VersionedProtocol: ClientProtocol or HRegionInterface.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 621

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line621>

        >

        > Want to adjust this message?

        Fixed.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1440

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1440>

        >

        > This set of protocols is covered by the synchronize held above? i.e. all access on protocols are under a block like this?

        Good point. Will fix.

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1449

        > <https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1449>

        >

        > Is there a difference here? We create an ISA only when we need it previously? Now we create it always? Is there a diff here?

        HServerAddress is deprecated. We are not going to get an ISA passed in. The protocol is cached so we don't need to create it many times.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6833
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-11 06:22:10, Michael Stack wrote: > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow. Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691 > < https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691 > > > I said it before but I like this code removal It is not used so I removed it. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 533 > < https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line533 > > > What about the EOFE you added today? Does that need to go in here anywhere? EOFE is an IOException. Good catch. I addressed it in HBASE-5621 , which is in testing now. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java, line 574 > < https://reviews.apache.org/r/4629/diff/2/?file=100872#file100872line574 > > > So, to be clear, if a client does not close a scanner now, how does it get cleaned up on the server? Does it live forever? In this case, the Leases daemon will expire the scanner and clean it up. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 184 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line184 > > > Rename to hbase.clientprotocol.class and CLIENT_PROTOCOL_CLASS. Seems more consistent? Will fix. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235 > < https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235 > > > Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think? > > Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this. I agree a ClientProtocol should have an HConnection. We can hide the HConnection. Given a region location, we can create a ClientProtocol which manages its own HConnection. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 187 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line187 > > > Rename too accordingly. Will fix. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 565 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line565 > > > This is pb VersionedProtocol? So far, it can be any VersionedProtocol: ClientProtocol or HRegionInterface. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 621 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line621 > > > Want to adjust this message? Fixed. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1440 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1440 > > > This set of protocols is covered by the synchronize held above? i.e. all access on protocols are under a block like this? Good point. Will fix. On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java, line 1449 > < https://reviews.apache.org/r/4629/diff/2/?file=100874#file100874line1449 > > > Is there a difference here? We create an ISA only when we need it previously? Now we create it always? Is there a diff here? HServerAddress is deprecated. We are not going to get an ISA passed in. The protocol is cached so we don't need to create it many times. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6833 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow.

        Jimmy Xiang wrote:

        Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response.

        Sorry, I don't get the last part? The response is too hard to make when the query is lots of types?

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691

        > <https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691>

        >

        > I said it before but I like this code removal

        Jimmy Xiang wrote:

        It is not used so I removed it.

        Great

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235

        > <https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235>

        >

        > Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think?

        >

        > Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this.

        Jimmy Xiang wrote:

        I agree a ClientProtocol should have an HConnection. We can hide the HConnection. Given a region location, we can create a ClientProtocol which manages its own HConnection.

        Sounds better

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6833
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-11 06:22:10, Michael Stack wrote: > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow. Jimmy Xiang wrote: Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response. Sorry, I don't get the last part? The response is too hard to make when the query is lots of types? On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java, line 691 > < https://reviews.apache.org/r/4629/diff/2/?file=100870#file100870line691 > > > I said it before but I like this code removal Jimmy Xiang wrote: It is not used so I removed it. Great On 2012-04-11 06:22:10, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HConnection.java, line 235 > < https://reviews.apache.org/r/4629/diff/2/?file=100873#file100873line235 > > > Does this feel right Jimmy? Seems odd asking an HConnection for a ClientProtocol? It feels like a ClientProtocol should have an HConnection? Or that we'd put together a ClientProtocol + an HConnection for it to use in a different object altogether. What you think? > > Maybe this is a question for another client implementation that we do later? But would be interested if you had any ideas on this. Jimmy Xiang wrote: I agree a ClientProtocol should have an HConnection. We can hide the HConnection. Given a region location, we can create a ClientProtocol which manages its own HConnection. Sounds better Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6833 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow.

        Jimmy Xiang wrote:

        Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response.

        Michael Stack wrote:

        Sorry, I don't get the last part? The response is too hard to make when the query is lots of types?

        Right, basically the code is not efficient.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6833
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-11 06:22:10, Michael Stack wrote: > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow. Jimmy Xiang wrote: Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response. Michael Stack wrote: Sorry, I don't get the last part? The response is too hard to make when the query is lots of types? Right, basically the code is not efficient. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6833 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6863
        -----------------------------------------------------------

        I'm halfway down on second page....

        src/main/java/org/apache/hadoop/hbase/client/HTable.java
        <https://reviews.apache.org/r/4629/#comment15282>

        Is there an EOFE story here too? You are dealing w/ that elsewhere?

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        <https://reviews.apache.org/r/4629/#comment15283>

        Yeah, we were going to try and get the data up from rpc layer

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        <https://reviews.apache.org/r/4629/#comment15284>

        Does this call need to return a Result or do we need to add a new method to support being able to get results as part of the call to open a scanner?

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java
        <https://reviews.apache.org/r/4629/#comment15285>

        Fix this text

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java
        <https://reviews.apache.org/r/4629/#comment15286>

        This is the right comment?

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java
        <https://reviews.apache.org/r/4629/#comment15287>

        Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early.

        • Michael

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6863 ----------------------------------------------------------- I'm halfway down on second page.... src/main/java/org/apache/hadoop/hbase/client/HTable.java < https://reviews.apache.org/r/4629/#comment15282 > Is there an EOFE story here too? You are dealing w/ that elsewhere? src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java < https://reviews.apache.org/r/4629/#comment15283 > Yeah, we were going to try and get the data up from rpc layer src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java < https://reviews.apache.org/r/4629/#comment15284 > Does this call need to return a Result or do we need to add a new method to support being able to get results as part of the call to open a scanner? src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java < https://reviews.apache.org/r/4629/#comment15285 > Fix this text src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java < https://reviews.apache.org/r/4629/#comment15286 > This is the right comment? src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java < https://reviews.apache.org/r/4629/#comment15287 > Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early. Michael On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6868
        -----------------------------------------------------------

        What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right?

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
        <https://reviews.apache.org/r/4629/#comment15293>

        Fix comment.

        Should you explain your intent for this class too? That HRegionServer will wither away? I think it woud be useful since it was not plain to me at first but after you explained it, I was on your side.

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
        <https://reviews.apache.org/r/4629/#comment15294>

        Shouldn't this implement Server and RegionServerServices?

        I would love to take the time to break this class up into smaller pieces but thats not for this JIRA. Can do that after this goes in.

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java
        <https://reviews.apache.org/r/4629/#comment15295>

        Should this class implement Closeable, Stoppable and Abortable?

        src/main/protobuf/Client.proto
        <https://reviews.apache.org/r/4629/#comment15297>

        Call this keyValueBytes then so method names give more of a clue as to what you'll be getting; save on the surprises?

        • Michael

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6868 ----------------------------------------------------------- What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right? src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java < https://reviews.apache.org/r/4629/#comment15293 > Fix comment. Should you explain your intent for this class too? That HRegionServer will wither away? I think it woud be useful since it was not plain to me at first but after you explained it, I was on your side. src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java < https://reviews.apache.org/r/4629/#comment15294 > Shouldn't this implement Server and RegionServerServices? I would love to take the time to break this class up into smaller pieces but thats not for this JIRA. Can do that after this goes in. src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java < https://reviews.apache.org/r/4629/#comment15295 > Should this class implement Closeable, Stoppable and Abortable? src/main/protobuf/Client.proto < https://reviews.apache.org/r/4629/#comment15297 > Call this keyValueBytes then so method names give more of a clue as to what you'll be getting; save on the surprises? Michael On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-11 06:22:10, Michael Stack wrote:

        > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow.

        Jimmy Xiang wrote:

        Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response.

        Michael Stack wrote:

        Sorry, I don't get the last part? The response is too hard to make when the query is lots of types?

        Jimmy Xiang wrote:

        Right, basically the code is not efficient.

        Whats your thoughts on making it efficient? Is it possible?

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6833
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-11 06:22:10, Michael Stack wrote: > I got as far as HTable. Still have more to go. This is some pretty great work Jimmy. This stuff is hard. One thought I was having that is a little related (but it can be safely ignored) is what if there were only one method on an HRegionServer and you gave it an array of Gets, Puts, Delete, Increment, pb messages and you just let the regionserver sort it out. is that even posssible? I'll do rest of review tomorrow. Jimmy Xiang wrote: Thanks for reviewing. As to one method to handle all actions, it is supported by the multi call. It is too complex to match the response. Michael Stack wrote: Sorry, I don't get the last part? The response is too hard to make when the query is lots of types? Jimmy Xiang wrote: Right, basically the code is not efficient. Whats your thoughts on making it efficient? Is it possible? Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6833 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 187

        > <https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187>

        >

        > Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early.

        Not for now, I can add it in a separate patch if it is ok with you.

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java, line 28

        > <https://reviews.apache.org/r/4629/diff/2/?file=100885#file100885line28>

        >

        > This is the right comment?

        Fixed.

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java, line 57

        > <https://reviews.apache.org/r/4629/diff/2/?file=100882#file100882line57>

        >

        > Fix this text

        Fixed.

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 190

        > <https://reviews.apache.org/r/4629/diff/2/?file=100876#file100876line190>

        >

        > Yeah, we were going to try and get the data up from rpc layer

        Yes, after changed pb Result to use key value bytes, we should be able to get the data now.

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/client/HTable.java, line 672

        > <https://reviews.apache.org/r/4629/diff/2/?file=100875#file100875line672>

        >

        > Is there an EOFE story here too? You are dealing w/ that elsewhere?

        EOFE is also an IOE. It should be covered as before.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6863
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 187 > < https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187 > > > Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early. Not for now, I can add it in a separate patch if it is ok with you. On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java, line 28 > < https://reviews.apache.org/r/4629/diff/2/?file=100885#file100885line28 > > > This is the right comment? Fixed. On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java, line 57 > < https://reviews.apache.org/r/4629/diff/2/?file=100882#file100882line57 > > > Fix this text Fixed. On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java, line 190 > < https://reviews.apache.org/r/4629/diff/2/?file=100876#file100876line190 > > > Yeah, we were going to try and get the data up from rpc layer Yes, after changed pb Result to use key value bytes, we should be able to get the data now. On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/client/HTable.java, line 672 > < https://reviews.apache.org/r/4629/diff/2/?file=100875#file100875line672 > > > Is there an EOFE story here too? You are dealing w/ that elsewhere? EOFE is also an IOE. It should be covered as before. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6863 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right?

        Could you please check it in if all tests pass? It can save me some rebasing/merging efforts.

        I can add unit tests for the new protobuf utility in a separate patch. I do have my own setup to commit my changes. I am also working on HBASE-5621.
        I can resolve other issues in HBASE-5621, or separate patches.

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > src/main/protobuf/Client.proto, line 56

        > <https://reviews.apache.org/r/4629/diff/2/?file=100900#file100900line56>

        >

        > Call this keyValueBytes then so method names give more of a clue as to what you'll be getting; save on the surprises?

        Sure, will fix.

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 143

        > <https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143>

        >

        > Should this class implement Closeable, Stoppable and Abortable?

        RegionServerServices defined Stoppable, Abortable. Now RegionServer implements RegionServerServices.
        Not sure about Closeable.

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98

        > <https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line98>

        >

        > Shouldn't this implement Server and RegionServerServices?

        >

        > I would love to take the time to break this class up into smaller pieces but thats not for this JIRA. Can do that after this goes in.

        RegionServer now implements RegionServerServices which extends Server.

        It will be great to break this class up.

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 94

        > <https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line94>

        >

        > Fix comment.

        >

        > Should you explain your intent for this class too? That HRegionServer will wither away? I think it woud be useful since it was not plain to me at first but after you explained it, I was on your side.

        Sure, added comments. Thanks.

        • Jimmy

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6868
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-12 04:50:15, Michael Stack wrote: > What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right? Could you please check it in if all tests pass? It can save me some rebasing/merging efforts. I can add unit tests for the new protobuf utility in a separate patch. I do have my own setup to commit my changes. I am also working on HBASE-5621 . I can resolve other issues in HBASE-5621 , or separate patches. On 2012-04-12 04:50:15, Michael Stack wrote: > src/main/protobuf/Client.proto, line 56 > < https://reviews.apache.org/r/4629/diff/2/?file=100900#file100900line56 > > > Call this keyValueBytes then so method names give more of a clue as to what you'll be getting; save on the surprises? Sure, will fix. On 2012-04-12 04:50:15, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 143 > < https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143 > > > Should this class implement Closeable, Stoppable and Abortable? RegionServerServices defined Stoppable, Abortable. Now RegionServer implements RegionServerServices. Not sure about Closeable. On 2012-04-12 04:50:15, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 98 > < https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line98 > > > Shouldn't this implement Server and RegionServerServices? > > I would love to take the time to break this class up into smaller pieces but thats not for this JIRA. Can do that after this goes in. RegionServer now implements RegionServerServices which extends Server. It will be great to break this class up. On 2012-04-12 04:50:15, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 94 > < https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line94 > > > Fix comment. > > Should you explain your intent for this class too? That HRegionServer will wither away? I think it woud be useful since it was not plain to me at first but after you explained it, I was on your side. Sure, added comments. Thanks. Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6868 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-12 00:03:44, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 187

        > <https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187>

        >

        > Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early.

        Jimmy Xiang wrote:

        Not for now, I can add it in a separate patch if it is ok with you.

        Yes. Make it a blocker though. I see unit tests finding bugs in this stuff.

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6863
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-12 00:03:44, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java, line 187 > < https://reviews.apache.org/r/4629/diff/2/?file=100889#file100889line187 > > > Do these have unit tests? Looks like it would be easy enough to add and could find some nasty bugs early. Jimmy Xiang wrote: Not for now, I can add it in a separate patch if it is ok with you. Yes. Make it a blocker though. I see unit tests finding bugs in this stuff. Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6863 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right?

        Jimmy Xiang wrote:

        Could you please check it in if all tests pass? It can save me some rebasing/merging efforts.

        I can add unit tests for the new protobuf utility in a separate patch. I do have my own setup to commit my changes. I am also working on HBASE-5621.

        I can resolve other issues in HBASE-5621, or separate patches.

        I can. You going to do another version to address above feedback?

        On 2012-04-12 04:50:15, Michael Stack wrote:

        > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 143

        > <https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143>

        >

        > Should this class implement Closeable, Stoppable and Abortable?

        Jimmy Xiang wrote:

        RegionServerServices defined Stoppable, Abortable. Now RegionServer implements RegionServerServices.

        Not sure about Closeable.

        Good

        • Michael

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/#review6868
        -----------------------------------------------------------

        On 2012-04-07 21:30:32, Jimmy Xiang wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/4629/

        -----------------------------------------------------------

        (Updated 2012-04-07 21:30:32)

        Review request for hbase.

        Summary

        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.

        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs

        -----

        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9

        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa

        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017

        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086

        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce

        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f

        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3

        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad

        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab

        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489

        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393

        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717

        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0

        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457

        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c

        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de

        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310

        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d

        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d

        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f

        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION

        src/main/protobuf/Admin.proto PRE-CREATION

        src/main/protobuf/Client.proto PRE-CREATION

        src/main/protobuf/RegionAdmin.proto c64d68b

        src/main/protobuf/RegionClient.proto 358382b

        src/main/protobuf/hbase.proto da78788

        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc

        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468

        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc

        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3

        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8

        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2

        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83

        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5

        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989

        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing

        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - On 2012-04-12 04:50:15, Michael Stack wrote: > What you want to do Jimmy? On next iteration, if all tests pass, should we check it in? I think its missing unit tests for the new protobuf utility. We should add those. I'd be fine adding them in a separate patch if that would suit you. This thing is already massive. We could commit to a branch too but that probably doesn't get us far; you have your own setup where you are to which you can commit, right? Jimmy Xiang wrote: Could you please check it in if all tests pass? It can save me some rebasing/merging efforts. I can add unit tests for the new protobuf utility in a separate patch. I do have my own setup to commit my changes. I am also working on HBASE-5621 . I can resolve other issues in HBASE-5621 , or separate patches. I can. You going to do another version to address above feedback? On 2012-04-12 04:50:15, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java, line 143 > < https://reviews.apache.org/r/4629/diff/2/?file=100898#file100898line143 > > > Should this class implement Closeable, Stoppable and Abortable? Jimmy Xiang wrote: RegionServerServices defined Stoppable, Abortable. Now RegionServer implements RegionServerServices. Not sure about Closeable. Good Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/#review6868 ----------------------------------------------------------- On 2012-04-07 21:30:32, Jimmy Xiang wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-07 21:30:32) Review request for hbase. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs ----- src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 97293aa src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 1316457 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java 19ae18c src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 8a61f7d src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/
        -----------------------------------------------------------

        (Updated 2012-04-12 17:25:51.153610)

        Review request for hbase.

        Changes
        -------

        Addressed Stack's comments.

        Summary
        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.
        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs (updated)


        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9
        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 3167f23
        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017
        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086
        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce
        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f
        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3
        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad
        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab
        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489
        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393
        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717
        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0
        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 0573c68
        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java b71ae66
        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de
        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9b34e61
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d
        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f
        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION
        src/main/protobuf/Admin.proto PRE-CREATION
        src/main/protobuf/Client.proto PRE-CREATION
        src/main/protobuf/RegionAdmin.proto c64d68b
        src/main/protobuf/RegionClient.proto 358382b
        src/main/protobuf/hbase.proto da78788
        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc
        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468
        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc
        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3
        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8
        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2
        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83
        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5
        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989
        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing
        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-12 17:25:51.153610) Review request for hbase. Changes ------- Addressed Stack's comments. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs (updated) src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 3167f23 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 0573c68 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java b71ae66 src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9b34e61 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522452/hbase-5620_v3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 30 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522452/hbase-5620_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1496//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Minor code change. Fixed a bug.

        Show
        Jimmy Xiang added a comment - Minor code change. Fixed a bug.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/4629/
        -----------------------------------------------------------

        (Updated 2012-04-12 22:33:53.615686)

        Review request for hbase.

        Changes
        -------

        Minor code change. Fixed a bug.

        Summary
        -------

        This is the client protocol part of region interface. The admin protocol part will be done in a different jira.

        The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface

        Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras.

        The existing client interface is not changed so that we don't break existing clients.

        This addresses bug HBASE-5620.
        https://issues.apache.org/jira/browse/HBASE-5620

        Diffs (updated)


        src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9
        src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 3167f23
        src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017
        src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086
        src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce
        src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f
        src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3
        src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad
        src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab
        src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489
        src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393
        src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717
        src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0
        src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 0573c68
        src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java b71ae66
        src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de
        src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310
        src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9b34e61
        src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d
        src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f
        src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION
        src/main/protobuf/Admin.proto PRE-CREATION
        src/main/protobuf/Client.proto PRE-CREATION
        src/main/protobuf/RegionAdmin.proto c64d68b
        src/main/protobuf/RegionClient.proto 358382b
        src/main/protobuf/hbase.proto da78788
        src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc
        src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468
        src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc
        src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3
        src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8
        src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2
        src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83
        src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5
        src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989
        src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b

        Diff: https://reviews.apache.org/r/4629/diff

        Testing
        -------

        "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again.

        Also tested it on a real cluster with ycsb and bigtop.

        Thanks,

        Jimmy

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4629/ ----------------------------------------------------------- (Updated 2012-04-12 22:33:53.615686) Review request for hbase. Changes ------- Minor code change. Fixed a bug. Summary ------- This is the client protocol part of region interface. The admin protocol part will be done in a different jira. The HRegionInterface is still there since the admin part is not done yet. The other reason is that in case some people still wants the old interface Filters, comparators and coprocessor parameters are still Writable. They will be addressed in different jiras. The existing client interface is not changed so that we don't break existing clients. This addresses bug HBASE-5620 . https://issues.apache.org/jira/browse/HBASE-5620 Diffs (updated) src/main/java/org/apache/hadoop/hbase/catalog/MetaReader.java 0129ee9 src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java 3167f23 src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 16e4017 src/main/java/org/apache/hadoop/hbase/client/HConnection.java 5d43086 src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 0b783ce src/main/java/org/apache/hadoop/hbase/client/HTable.java aa7652f src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java 9903df3 src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java ddcf9ad src/main/java/org/apache/hadoop/hbase/filter/ParseConstants.java 1acbdab src/main/java/org/apache/hadoop/hbase/io/HbaseObjectWritable.java cbfa489 src/main/java/org/apache/hadoop/hbase/io/TimeRange.java d135393 src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java 05ae717 src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java f1f06b0 src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java 0573c68 src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java b71ae66 src/main/java/org/apache/hadoop/hbase/protobuf/AdminProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java 2eb57de src/main/java/org/apache/hadoop/hbase/protobuf/RequestConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/ResponseConverter.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/AdminProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/ClientProtos.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/protobuf/generated/HBaseProtos.java 4026da0 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionAdminProtos.java 2169310 src/main/java/org/apache/hadoop/hbase/protobuf/generated/RegionClientProtos.java b36a9c0 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 9b34e61 src/main/java/org/apache/hadoop/hbase/regionserver/HRegionThriftServer.java 703e73d src/main/java/org/apache/hadoop/hbase/regionserver/Leases.java b520f3f src/main/java/org/apache/hadoop/hbase/regionserver/RegionServer.java PRE-CREATION src/main/protobuf/Admin.proto PRE-CREATION src/main/protobuf/Client.proto PRE-CREATION src/main/protobuf/RegionAdmin.proto c64d68b src/main/protobuf/RegionClient.proto 358382b src/main/protobuf/hbase.proto da78788 src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java c7284dc src/test/java/org/apache/hadoop/hbase/catalog/TestMetaReaderEditorNoCluster.java 0042468 src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java e34d8bc src/test/java/org/apache/hadoop/hbase/io/TestHbaseObjectWritable.java f2f8ee3 src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java 41616c8 src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java 6ed4ba2 src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java b4dcb83 src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java ceca6f5 src/test/java/org/apache/hadoop/hbase/regionserver/OOMERegionServer.java cac2989 src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionServerBulkLoad.java a1bf73b Diff: https://reviews.apache.org/r/4629/diff Testing ------- "mvn -PrunAllTests clean test" is green, except some flaky tests which need to run again. Also tested it on a real cluster with ycsb and bigtop. Thanks, Jimmy
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522499/hbase-5620_v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 30 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
        org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
        org.apache.hadoop.hbase.regionserver.wal.TestHLog
        org.apache.hadoop.hbase.replication.TestMasterReplication

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522499/hbase-5620_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.regionserver.wal.TestHLog org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1501//console This message is automatically generated.
        Hide
        stack added a comment -

        Retrying Jimmy's patch

        Show
        stack added a comment - Retrying Jimmy's patch
        Hide
        stack added a comment -

        Retry against hadoopqa

        Show
        stack added a comment - Retry against hadoopqa
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522531/hbase-5620_v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 30 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
        org.apache.hadoop.hbase.replication.TestReplication
        org.apache.hadoop.hbase.mapreduce.TestWALPlayer
        org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit
        org.apache.hadoop.hbase.replication.TestMultiSlaveReplication
        org.apache.hadoop.hbase.regionserver.wal.TestHLog
        org.apache.hadoop.hbase.replication.TestMasterReplication

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522531/hbase-5620_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.mapreduce.TestWALPlayer org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit org.apache.hadoop.hbase.replication.TestMultiSlaveReplication org.apache.hadoop.hbase.regionserver.wal.TestHLog org.apache.hadoop.hbase.replication.TestMasterReplication Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1508//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        I will take a look at the test failures.

        Show
        Jimmy Xiang added a comment - I will take a look at the test failures.
        Hide
        stack added a comment -

        Retry after backing out wal compression patch that was breaking a few tests. Will narrow the set of tests that need looking at.

        Show
        stack added a comment - Retry after backing out wal compression patch that was breaking a few tests. Will narrow the set of tests that need looking at.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522579/hbase-5620_v4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 30 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.mapreduce.TestWALPlayer

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522579/hbase-5620_v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 30 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestWALPlayer Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1511//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        TestWALPlayer passed for me. I didn't have the latest from trunk?

        Show
        Jimmy Xiang added a comment - TestWALPlayer passed for me. I didn't have the latest from trunk?
        Hide
        stack added a comment -

        It passed for me. Let me commit this monster.

        Show
        stack added a comment - It passed for me. Let me commit this monster.
        Hide
        stack added a comment -

        I think src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java is in wrong package. Ditto for AdminProtocol. What you think Jimmy? Should we move them? Where should they go? At top level? Or into client package?

        Show
        stack added a comment - I think src/main/java/org/apache/hadoop/hbase/protobuf/ClientProtocol.java is in wrong package. Ditto for AdminProtocol. What you think Jimmy? Should we move them? Where should they go? At top level? Or into client package?
        Hide
        stack added a comment -

        Mind opening new issues Jimmy to do outstanding work like unit tests?

        Show
        stack added a comment - Mind opening new issues Jimmy to do outstanding work like unit tests?
        Hide
        stack added a comment -

        Committed to trunk. Thats some pretty heavy lifting you got going on in here Jimmy. Good stuff.

        Show
        stack added a comment - Committed to trunk. Thats some pretty heavy lifting you got going on in here Jimmy. Good stuff.
        Hide
        Ted Yu added a comment -

        Looks like trunk build failed with:

        [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase: Too many unapproved licenses: 1 -> [Help 1]
        org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check (default) on project hbase: Too many unapproved licenses: 1
        
        Show
        Ted Yu added a comment - Looks like trunk build failed with: [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check ( default ) on project hbase: Too many unapproved licenses: 1 -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.rat:apache-rat-plugin:0.8:check ( default ) on project hbase: Too many unapproved licenses: 1
        Hide
        Jimmy Xiang added a comment -

        @Stack, I will move ClientProtocol.java and AdminProtocol.java to top level in HBASE-5621 since they are common. I added HBASE-5785 to track the unit test issue.

        @Ted, can I check the licenses without doing a release build?

        Show
        Jimmy Xiang added a comment - @Stack, I will move ClientProtocol.java and AdminProtocol.java to top level in HBASE-5621 since they are common. I added HBASE-5785 to track the unit test issue. @Ted, can I check the licenses without doing a release build?
        Hide
        Ted Yu added a comment -

        Here is the command Jenkins used:

        [trunk] $ /home/hudson/tools/maven/latest3/bin/mvn -e -X clean -Dmaven.test.redirectTestOutputToFile=true site install assembly:single -DskipITs -Prelease
        
        Show
        Ted Yu added a comment - Here is the command Jenkins used: [trunk] $ /home/hudson/tools/maven/latest3/bin/mvn -e -X clean -Dmaven.test.redirectTestOutputToFile= true site install assembly:single -DskipITs -Prelease
        Hide
        Jimmy Xiang added a comment -

        I ran Apache Rat check like mvn apache-rat:check, and it is ok.

        Show
        Jimmy Xiang added a comment - I ran Apache Rat check like mvn apache-rat:check, and it is ok.
        Hide
        stack added a comment -

        I did too and trunk is also no longer complaining. The rat test is a PITA. There was probably some deitrus laying around that it picked up. I modified the trunk build to keep the rat.txt report next time.

        @Jimmy I think top-level is better than where it currently is. What other Protocols would go up to the top level? None I suppose. I suppose they should be in client package but its a little perverse having the the client stuff reaching into zk and util and protobuf...

        Show
        stack added a comment - I did too and trunk is also no longer complaining. The rat test is a PITA. There was probably some deitrus laying around that it picked up. I modified the trunk build to keep the rat.txt report next time. @Jimmy I think top-level is better than where it currently is. What other Protocols would go up to the top level? None I suppose. I suppose they should be in client package but its a little perverse having the the client stuff reaching into zk and util and protobuf...
        Hide
        Jimmy Xiang added a comment -

        @Stack, thanks a lot!

        I moved them to top-level in HBase-5621 and posted a review request. Could you please review?
        I am ok to move them to client package.

        Show
        Jimmy Xiang added a comment - @Stack, thanks a lot! I moved them to top-level in HBase-5621 and posted a review request. Could you please review? I am ok to move them to client package.
        Hide
        stack added a comment -

        @Jimmy I think this is the biggest patch ever applied to HBase. Congrats!

        Show
        stack added a comment - @Jimmy I think this is the biggest patch ever applied to HBase. Congrats!
        Hide
        stack added a comment -

        @Jimmy I think this is the biggest patch ever applied to HBase. Congrats!

        Show
        stack added a comment - @Jimmy I think this is the biggest patch ever applied to HBase. Congrats!
        Hide
        Ted Yu added a comment -

        As you can see in https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-security/171/, there were 12 more test failures.

        To narrow the checkin that caused additional test failures, I locally backed out HBASE-5684 and HBASE-5747. TestAccessControlFilter#testQualifierAccess still failed with access permission error.
        After backing out this JIRA, the test passed.

        Show
        Ted Yu added a comment - As you can see in https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-security/171/ , there were 12 more test failures. To narrow the checkin that caused additional test failures, I locally backed out HBASE-5684 and HBASE-5747 . TestAccessControlFilter#testQualifierAccess still failed with access permission error. After backing out this JIRA, the test passed.
        Hide
        Jimmy Xiang added a comment -

        @Stack, thanks.

        @Ted, I am looking into it now.

        Show
        Jimmy Xiang added a comment - @Stack, thanks. @Ted, I am looking into it now.
        Hide
        Jimmy Xiang added a comment -

        hbase-5620-sec.patch is to fix the security unit test failures.

        When I ran all tests, I forgot the security part, which has a different RPC engine.

        Show
        Jimmy Xiang added a comment - hbase-5620-sec.patch is to fix the security unit test failures. When I ran all tests, I forgot the security part, which has a different RPC engine.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12522687/hbase-5620-sec.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12522687/hbase-5620-sec.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 3 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1527//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        TestForceCacheImportantBlocks is green for me.

        Show
        Jimmy Xiang added a comment - TestForceCacheImportantBlocks is green for me.
        Hide
        Ted Yu added a comment -

        Addendum makes sense.

        TestAccessControlFilter passes with the addendum.
        Whole security test suite should be run.

        Show
        Ted Yu added a comment - Addendum makes sense. TestAccessControlFilter passes with the addendum. Whole security test suite should be run.
        Hide
        Jimmy Xiang added a comment -

        Thanks for reviewing. Both regular test suite and security test suite are green for me. I mean all tests in the suite.

        Show
        Jimmy Xiang added a comment - Thanks for reviewing. Both regular test suite and security test suite are green for me. I mean all tests in the suite.
        Hide
        Todd Lipcon added a comment -

        Did anyone do any benchmarks here? It concerns me to have this committed without even knowing how it affects performance.

        Show
        Todd Lipcon added a comment - Did anyone do any benchmarks here? It concerns me to have this committed without even knowing how it affects performance.
        Hide
        Jimmy Xiang added a comment -

        I did some testing with YCSB (mostly inserts). It gave me better performance for the patch which was a surprise to me.
        I will do some read-only testing with YCSB too.

        Show
        Jimmy Xiang added a comment - I did some testing with YCSB (mostly inserts). It gave me better performance for the patch which was a surprise to me. I will do some read-only testing with YCSB too.
        Hide
        Ted Yu added a comment -

        @Stack:
        Do you think the addendum is ready to be integrated ?

        Show
        Ted Yu added a comment - @Stack: Do you think the addendum is ready to be integrated ?
        Hide
        stack added a comment -

        @Jimmy So every invocation will throw an exception?

        +        // For protobuf protocols, ServiceException is expected
        

        Whats the Set in Invocation doing? You add it but don't seem to access it?

        I like the removal of a call method down through the rpc stack

        Show
        stack added a comment - @Jimmy So every invocation will throw an exception? + // For protobuf protocols, ServiceException is expected Whats the Set in Invocation doing? You add it but don't seem to access it? I like the removal of a call method down through the rpc stack
        Hide
        Jimmy Xiang added a comment -

        @Stack, not every invocation will throw an exception. In case it throws an exception, it should be a ServiceException for pb. It used to be IOException. Without the change, for pb calls, it won't get a ServiceException in case something goes wrong. It gets an undeclared exception
        with the cause to be an IOE, and the upper layer doesn't know how to handle it.

        The Set in Invocation is used to decide if a protocol a pb one, so ServiceException should be used. I put it there because it is
        used for both WritableRpcEngine and SecureRpcEngine.

        Show
        Jimmy Xiang added a comment - @Stack, not every invocation will throw an exception. In case it throws an exception, it should be a ServiceException for pb. It used to be IOException. Without the change, for pb calls, it won't get a ServiceException in case something goes wrong. It gets an undeclared exception with the cause to be an IOE, and the upper layer doesn't know how to handle it. The Set in Invocation is used to decide if a protocol a pb one, so ServiceException should be used. I put it there because it is used for both WritableRpcEngine and SecureRpcEngine.
        Hide
        stack added a comment -

        I made HBASE-5810 to apply this Jimmy. Good stuff.

        Show
        stack added a comment - I made HBASE-5810 to apply this Jimmy. Good stuff.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2781 (See https://builds.apache.org/job/HBase-TRUNK/2781/)
        HBASE-5810 HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum (Revision 1327629)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2781 (See https://builds.apache.org/job/HBase-TRUNK/2781/ ) HBASE-5810 HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum (Revision 1327629) Result = FAILURE stack : Files : /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #175 (See https://builds.apache.org/job/HBase-TRUNK-security/175/)
        HBASE-5810 HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum (Revision 1327629)

        Result = FAILURE
        stack :
        Files :

        • /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java
        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #175 (See https://builds.apache.org/job/HBase-TRUNK-security/175/ ) HBASE-5810 HBASE-5620 Convert the client protocol of HRegionInterface to PB addendum (Revision 1327629) Result = FAILURE stack : Files : /hbase/trunk/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureRpcEngine.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ScannerCallable.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/ServerCallable.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/ExecRPCInvoker.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/Invocation.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/RpcEngine.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java
        Hide
        Jimmy Xiang added a comment -

        I did some testing with YCSB. I loaded 9000 rows to a table and read (Get) from it for 2000000 times.
        The idea is to have less data so that all are in memory, so as to test the RPC overhead.

        The performance number changes. On average, the average read latency is 335us without the patch, 362us with the patch.
        It is about 27us (8%) degrade.

        Show
        Jimmy Xiang added a comment - I did some testing with YCSB. I loaded 9000 rows to a table and read (Get) from it for 2000000 times. The idea is to have less data so that all are in memory, so as to test the RPC overhead. The performance number changes. On average, the average read latency is 335us without the patch, 362us with the patch. It is about 27us (8%) degrade.
        Hide
        Todd Lipcon added a comment -

        Hey Jimmy. Any difference on throughput?

        Show
        Todd Lipcon added a comment - Hey Jimmy. Any difference on throughput?
        Hide
        Jimmy Xiang added a comment -

        Yes, the throughput is also degraded. Without the patch, it is 2941 ops/sec; with the patch it is 2725 ops/sec, about 7.4% degrade.

        Show
        Jimmy Xiang added a comment - Yes, the throughput is also degraded. Without the patch, it is 2941 ops/sec; with the patch it is 2725 ops/sec, about 7.4% degrade.
        Hide
        Todd Lipcon added a comment -

        That sounds like you only ran YCSB with 1 thread...? 1 sec / 2725 = 366us.

        Show
        Todd Lipcon added a comment - That sounds like you only ran YCSB with 1 thread...? 1 sec / 2725 = 366us.
        Hide
        Jimmy Xiang added a comment -

        You are right. I used just one thread. This can give us a clear picture on pb rpc overhead, right?

        Show
        Jimmy Xiang added a comment - You are right. I used just one thread. This can give us a clear picture on pb rpc overhead, right?
        Hide
        Ted Yu added a comment -

        Is there plan to adopt various measures to counter this 8% performance dip ?

        Show
        Ted Yu added a comment - Is there plan to adopt various measures to counter this 8% performance dip ?
        Hide
        Jimmy Xiang added a comment -

        Yes, I was thinking to eliminate the conversion layer if possible, probably starting from the server side.
        As to I performance test I did, I think the 27us overhead is more meaningful than the percentage number.

        Show
        Jimmy Xiang added a comment - Yes, I was thinking to eliminate the conversion layer if possible, probably starting from the server side. As to I performance test I did, I think the 27us overhead is more meaningful than the percentage number.
        Hide
        stack added a comment -

        Jimmy Xiang In ClientProtocol (and AdminProtocol, etc.), was there a particular reason for implementing BlockingInterface? See below:

        +/**
        + * Protocol that a HBase client uses to communicate with a region server.
        + */
        +@KerberosInfo(
        +  serverPrincipal = "hbase.regionserver.kerberos.principal")
        +@TokenInfo("HBASE_AUTH_TOKEN")
        +@InterfaceAudience.Public
        +@InterfaceStability.Evolving
        +public interface ClientProtocol extends
        +    ClientService.BlockingInterface, VersionedProtocol {
        +  public static final long VERSION = 1L;
        +}
        

        For the annotations, @TokenInfo, @KerberosInfo, what made you add these?

        Thanks Jimmy. Just trying to figure out how we arrived at what we have in our Protocols.

        Show
        stack added a comment - Jimmy Xiang In ClientProtocol (and AdminProtocol, etc.), was there a particular reason for implementing BlockingInterface? See below: +/** + * Protocol that a HBase client uses to communicate with a region server. + */ +@KerberosInfo( + serverPrincipal = "hbase.regionserver.kerberos.principal" ) +@TokenInfo( "HBASE_AUTH_TOKEN" ) +@InterfaceAudience.Public +@InterfaceStability.Evolving + public interface ClientProtocol extends + ClientService.BlockingInterface, VersionedProtocol { + public static final long VERSION = 1L; +} For the annotations, @TokenInfo, @KerberosInfo, what made you add these? Thanks Jimmy. Just trying to figure out how we arrived at what we have in our Protocols.
        Hide
        Jimmy Xiang added a comment -

        There is no special reason other than that our RPC engine uses blocking call.

        As to @TokenInfo, @KerberosInfo, they are inherited from the existing old HRegionInterface.

        Show
        Jimmy Xiang added a comment - There is no special reason other than that our RPC engine uses blocking call. As to @TokenInfo, @KerberosInfo, they are inherited from the existing old HRegionInterface.
        Hide
        stack added a comment -

        Jimmy Xiang BlockingInterface has references to RpcController in it, a facility we make no use of. For example, here is the get from the Interface:

              public org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse get(
                  com.google.protobuf.RpcController controller,
                  org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetRequest request)
                  throws com.google.protobuf.ServiceException;
        

        Whenever we use the method, we always pass null for the RpcController as in:

              GetResponse response = client.get(null, request);
        

        Were you thinking this was ok price to pay for not redefining the Interface minus mention of RpcProtocol? Thanks.

        Show
        stack added a comment - Jimmy Xiang BlockingInterface has references to RpcController in it, a facility we make no use of. For example, here is the get from the Interface: public org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetResponse get( com.google.protobuf.RpcController controller, org.apache.hadoop.hbase.protobuf.generated.ClientProtos.GetRequest request) throws com.google.protobuf.ServiceException; Whenever we use the method, we always pass null for the RpcController as in: GetResponse response = client.get( null , request); Were you thinking this was ok price to pay for not redefining the Interface minus mention of RpcProtocol? Thanks.
        Hide
        Jimmy Xiang added a comment -

        I thought about that. It is a clean choice to redefine the interface. I was wondering we may be able to use RpcController somehow in the future if we need to do something to the RPC engine.

        @Stack, if we are sure we are not going to use RpcController, we can redefine the interface and now is the time.

        Show
        Jimmy Xiang added a comment - I thought about that. It is a clean choice to redefine the interface. I was wondering we may be able to use RpcController somehow in the future if we need to do something to the RPC engine. @Stack, if we are sure we are not going to use RpcController, we can redefine the interface and now is the time.
        Hide
        stack added a comment -

        Jimmy Xiang I've been experimenting over in HBASE-6521 RpcController seems pretty useless though experimenting I ended up using it to carry extra payload in and out of the method on the server-side, not in client as here.

        VersionedProtocol is going away. We could just resort to raw BlockingInterfaces everywhere instead (though I do not think this will be work too cleanly since there is no common ancestor to BlockingInterface). We could redefine all methods to be what is in BlockingInterface minus this rpccontroller stuff – or, as you have it, just use what protoc generates as the Interface and be clear that the rpccontroller goes unused.

        I think we should go w/ the latter approach. Use raw blockinginterface where we can being clear rpccontroller is an unused field.

        Thanks for the answers Jimmy.

        Show
        stack added a comment - Jimmy Xiang I've been experimenting over in HBASE-6521 RpcController seems pretty useless though experimenting I ended up using it to carry extra payload in and out of the method on the server-side, not in client as here. VersionedProtocol is going away. We could just resort to raw BlockingInterfaces everywhere instead (though I do not think this will be work too cleanly since there is no common ancestor to BlockingInterface). We could redefine all methods to be what is in BlockingInterface minus this rpccontroller stuff – or, as you have it, just use what protoc generates as the Interface and be clear that the rpccontroller goes unused. I think we should go w/ the latter approach. Use raw blockinginterface where we can being clear rpccontroller is an unused field. Thanks for the answers Jimmy.
        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            Jimmy Xiang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development