Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.0
    • Fix Version/s: None
    • Component/s: Client, Coprocessors
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From Jesse's comment on dev:

      What I would like to propose is a simple interface that people can use to implement a 'constraint' (matching the classic database definition). This would help ease of adoption by helping HBase more easily check that box, help minimize code duplication across organizations, and lead to easier adoption.

      Essentially, people would implement a 'Constraint' interface for checking keys before they are put into a table. Puts that are valid get written to the table, but if not people can will throw an exception that gets propagated back to the client explaining why the put was invalid.

      Constraints would be set on a per-table basis and the user would be expected to ensure the jars containing the constraint are present on the machines serving that table.

      Yes, people could roll their own mechanism for doing this via coprocessors each time, but this would make it easier to do so, so you only have to implement a very minimal interface and not worry about the specifics.

      1. 4605.v7
        69 kB
        Ted Yu
      2. 4605-v6.txt
        73 kB
        Ted Yu
      3. constraint_as_cp.txt
        19 kB
        Jesse Yates
      4. java_Constraint_v2.patch
        8 kB
        Jesse Yates
      5. java_HBASE-4605_v1.patch
        80 kB
        Jesse Yates
      6. java_HBASE-4605_v2.patch
        80 kB
        Jesse Yates
      7. java_HBASE-4605_v3.patch
        79 kB
        Jesse Yates
      8. java_HBASE-4605_v5.patch
        73 kB
        Jesse Yates
      9. java_HBASE-4605_v7.patch
        75 kB
        Jesse Yates

        Issue Links

          Activity

          Jesse Yates created issue -
          Hide
          Jesse Yates added a comment -

          I've been thinking about how to go about implementing this and have two ways to go about it.

          Method 1:
          My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these

          This means people need to distribute the jars and set conf values similar to what they would have to do before, but we would handle making sure the implemented Constraints get run in the right place, and propagate the errors (e.g. ConstraintFailedExecption) back to the users.

          Pluses:
          1) Allows users to get back multiple reasons why a put failed.
          2) Allows a ConstraintImpl to be a subclass of any arbitrary class and not bound to some abstract constraint.
          3) People don't have to worry about it being a coprocessor - it is notionally divorced.

          Minuses:
          1) Requires changing a bunch of code in HTableDescriptor and essentially duplicating a lot the checking/setting already done for coprocessors. This can be gotten around by generalizing the mechanism for storing classes in the HTD.

          Method 2 (already implemented, patch coming):
          Add superclass AbstractConstraint which only exposes a check(Put) method. It is actually a Coprocoessor which is loaded, processes the check and then returns the error to the client (wrapped in an IOException) on failure.
          Pluses:
          1) We don't have to implement any new mechanisms for specifying the constraint, people just have to add it as a coprocessor.
          Minuses:
          1) It could be confusing since with this mechanism, you just want people to think in terms of Constraints, not coprocessors
          2) You are bound to extending the AbstractCoprocessor, not just implementing the interface
          3) If just one constraint fails, then the put is rejected, so you can't find out all the reasons it would fail (useful if cleaning data).
          4) It doesn't really help 'simplify' the use of HBase. In fact, it increases the complexity.

          Show
          Jesse Yates added a comment - I've been thinking about how to go about implementing this and have two ways to go about it. Method 1: My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these This means people need to distribute the jars and set conf values similar to what they would have to do before, but we would handle making sure the implemented Constraints get run in the right place, and propagate the errors (e.g. ConstraintFailedExecption) back to the users. Pluses: 1) Allows users to get back multiple reasons why a put failed. 2) Allows a ConstraintImpl to be a subclass of any arbitrary class and not bound to some abstract constraint. 3) People don't have to worry about it being a coprocessor - it is notionally divorced. Minuses: 1) Requires changing a bunch of code in HTableDescriptor and essentially duplicating a lot the checking/setting already done for coprocessors. This can be gotten around by generalizing the mechanism for storing classes in the HTD. Method 2 (already implemented, patch coming): Add superclass AbstractConstraint which only exposes a check(Put) method. It is actually a Coprocoessor which is loaded, processes the check and then returns the error to the client (wrapped in an IOException) on failure. Pluses: 1) We don't have to implement any new mechanisms for specifying the constraint, people just have to add it as a coprocessor. Minuses: 1) It could be confusing since with this mechanism, you just want people to think in terms of Constraints, not coprocessors 2) You are bound to extending the AbstractCoprocessor, not just implementing the interface 3) If just one constraint fails, then the put is rejected, so you can't find out all the reasons it would fail (useful if cleaning data). 4) It doesn't really help 'simplify' the use of HBase. In fact, it increases the complexity.
          Hide
          Gary Helmling added a comment -

          My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these

          I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses...

          For setting the constraint implementations to be applied per table, I agree that using table attributes is probably easiest. But I don't see why we need to modify HTableDescriptor to enable this? We currently have HBASE-4554, which is looking to enable setting table attributes for coprocessors from the shell. It seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code?

          Show
          Gary Helmling added a comment - My idea is to write a ConstraintProcessor that is a system level CP with system wide support for setting constraints on a table. this requires add 'top-level' configuration values that the user would set for constraints to run (which would be ordered like coprocessors), but they would just implement the 'Constraint' interface. This means modifying HTD and the shell to enable all these I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses... For setting the constraint implementations to be applied per table, I agree that using table attributes is probably easiest. But I don't see why we need to modify HTableDescriptor to enable this? We currently have HBASE-4554 , which is looking to enable setting table attributes for coprocessors from the shell. It seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code?
          Hide
          Nicolas Liochon added a comment -

          It would be a very useful features.

          For what it worth, I am quite fine with method 2.

          Some points I found useful in the SQL engines:

          • capacity to disable the constraints for a connection (basically because yoy're doing a big set of operation and you want the best possible performances
          • capacity to disable globally (upgrade)

          I also wonder how we should manage the evolution of the constraint. On an SQL db, there is only one possible set; considering the amount of data, the traditional upgrade if the traditional sql db could not be possible here; so managing evolution of constraint would make sense.

          A comment as well: in a SQL system, the constraints are linked to the transaction: it's checked once the transaction is committed. This is important for cross table constraint checks, as there is no transaction between tables in HBase. I would tend to believe that's mainly a question of documentation (insert in the right order), but it's something to remember anyway (especially as you want to duplicate the relationships with HBase more than with a sql db)...

          I don't know the status of HCatalog, but I think the HCatalog schema will be transformable as HBase constraints, adding value to the two of them...

          Show
          Nicolas Liochon added a comment - It would be a very useful features. For what it worth, I am quite fine with method 2. Some points I found useful in the SQL engines: capacity to disable the constraints for a connection (basically because yoy're doing a big set of operation and you want the best possible performances capacity to disable globally (upgrade) I also wonder how we should manage the evolution of the constraint. On an SQL db, there is only one possible set; considering the amount of data, the traditional upgrade if the traditional sql db could not be possible here; so managing evolution of constraint would make sense. A comment as well: in a SQL system, the constraints are linked to the transaction: it's checked once the transaction is committed. This is important for cross table constraint checks, as there is no transaction between tables in HBase. I would tend to believe that's mainly a question of documentation (insert in the right order), but it's something to remember anyway (especially as you want to duplicate the relationships with HBase more than with a sql db)... I don't know the status of HCatalog, but I think the HCatalog schema will be transformable as HBase constraints, adding value to the two of them...
          Hide
          Jesse Yates added a comment -

          @gary:

          I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses...

          That is exactly what I was thinking for the "top-level" implementation

          seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code

          +1

          Right now coprocessors have a special syntax for loading on table level, which feels kind of clunky to do by hand (specifying COPROCESSOR$). I feel like we could definitely help enable setting values with a more concrete syntax (like a setCoprocessor method that we have on the HTableDescriptor now), which should handle the numbering, etc.

          So using an abstract version of the stuff from 4554 would definitely help with that. I don't know if we can just the use update shell though - we would probably need to update the java connection as well.

          Right now the code for storing things in the conf would be fine, we just need to abstract it a little bit, so it would look something like:

          public void addCoprocessor(name){
           addProcessingElement("coprocessor$", name);}
          
          public void addConstriant(name){
           addProcessingElement("constriant$", name);}
          
          public void addProcessingElement(String tag, String value){
          ...//all the checking/add currently in addCoprocessor
          }
          

          @keywal:

          Since they are just table configuration values, turning them on/off will be relatively painless.

          Cross-table transactions is separate can of worms and really goes against the whole design paradigm of HBase (see discussion on dev about this). This would be optimized to do single table checking, though people could implement cross table checks at serious cost (and later we can build in more optimized mechanisms if it is a common thing people do).

          HCatalog schema will be transformable as HBase constraints, adding value to the two of them...

          That should be super simple, it would just take a simple tool to create the corresponding constraints. I would use constraints to enforce things like data sanitation, rather than schema enforcement (its the last ditch barrier to things going into a table properly, since shipping things across the wire is expensive), which should be done client side, but it could definitely work.

          Show
          Jesse Yates added a comment - @gary: I like the idea of using a system level coprocessor with a minimal extension interface for the checks to be performed. For the actual interface, you could even use Predicate from the google guava lib, or have Constraint just be a named interface that extends Predicate<Put>. Not critical, but plugging in to a standard interface instead of doing a one-off may enable future uses... That is exactly what I was thinking for the "top-level" implementation seems like we could make that sufficiently generic to enable both the coprocessors case and this with just changes to the shell code +1 Right now coprocessors have a special syntax for loading on table level, which feels kind of clunky to do by hand (specifying COPROCESSOR$). I feel like we could definitely help enable setting values with a more concrete syntax (like a setCoprocessor method that we have on the HTableDescriptor now), which should handle the numbering, etc. So using an abstract version of the stuff from 4554 would definitely help with that. I don't know if we can just the use update shell though - we would probably need to update the java connection as well. Right now the code for storing things in the conf would be fine, we just need to abstract it a little bit, so it would look something like: public void addCoprocessor(name){ addProcessingElement( "coprocessor$" , name);} public void addConstriant(name){ addProcessingElement( "constriant$" , name);} public void addProcessingElement( String tag, String value){ ... //all the checking/add currently in addCoprocessor } @keywal: Since they are just table configuration values, turning them on/off will be relatively painless. Cross-table transactions is separate can of worms and really goes against the whole design paradigm of HBase (see discussion on dev about this). This would be optimized to do single table checking, though people could implement cross table checks at serious cost (and later we can build in more optimized mechanisms if it is a common thing people do). HCatalog schema will be transformable as HBase constraints, adding value to the two of them... That should be super simple, it would just take a simple tool to create the corresponding constraints. I would use constraints to enforce things like data sanitation, rather than schema enforcement (its the last ditch barrier to things going into a table properly, since shipping things across the wire is expensive), which should be done client side, but it could definitely work.
          Hide
          Andrew Purtell added a comment -

          +1 on choice #1, with generalization of argument passing to coprocessors via table attributes. HBASE-4048 and HBASE-4554 are a start there.

          Show
          Andrew Purtell added a comment - +1 on choice #1, with generalization of argument passing to coprocessors via table attributes. HBASE-4048 and HBASE-4554 are a start there.
          Hide
          Jesse Yates added a comment -

          Constraint implementation that is just added as a coprocessor. Not implemented as a Precondition for ease, though it could be ported over to that fairly easily. Basically, putting this up for posterity since the consensus seems to be pursuing #1 above.

          Also, is there a better way to pass back exceptions from coprocessors? Right now, the exception causes a retry which is a huge timeout problem

          Show
          Jesse Yates added a comment - Constraint implementation that is just added as a coprocessor. Not implemented as a Precondition for ease, though it could be ported over to that fairly easily. Basically, putting this up for posterity since the consensus seems to be pursuing #1 above. Also, is there a better way to pass back exceptions from coprocessors? Right now, the exception causes a retry which is a huge timeout problem
          Jesse Yates made changes -
          Field Original Value New Value
          Attachment java_Constraint_v2.patch [ 12499492 ]
          Hide
          Ted Yu added a comment -

          See the following from HBASE-4014 w.r.t. exceptions and coprocessors:

          The general gist here is to wrap each of

          {Master,RegionServer}

          CoprocessorHost's coprocessor call inside a

          "try

          { ... }

          catch (Throwable e)

          { handleCoprocessorThrowable(e) }

          "

          block.
          handleCoprocessorThrowable() is responsible for either passing 'e' along to the client (if 'e' is an IOException) or, otherwise, aborting the service (Regionserver or Master).

          Show
          Ted Yu added a comment - See the following from HBASE-4014 w.r.t. exceptions and coprocessors: The general gist here is to wrap each of {Master,RegionServer} CoprocessorHost's coprocessor call inside a "try { ... } catch (Throwable e) { handleCoprocessorThrowable(e) } " block. handleCoprocessorThrowable() is responsible for either passing 'e' along to the client (if 'e' is an IOException) or, otherwise, aborting the service (Regionserver or Master).
          Hide
          Jesse Yates added a comment -

          @Ted: yeah, I found that wrapping. However, tracing the execution back up, a simple put (or delete) ends up being called wrapped in a retry loop that tries the put every time a Throwable is received. So we end up with a RetriesExhaustedException, rather than the original exception being propagated back out. The path down for the put was HTable.put->doPut->flushCommits->HConnectionMananger.processBatchOfPuts, which does the retries. The end result is it takes minutes to do a commit because we are doing this checking and retrying, rather than failing out as the CP docs would imply. I feel like this could have serious impacts outside of this patch too..
          Unless I'm missing something?

          Show
          Jesse Yates added a comment - @Ted: yeah, I found that wrapping. However, tracing the execution back up, a simple put (or delete) ends up being called wrapped in a retry loop that tries the put every time a Throwable is received. So we end up with a RetriesExhaustedException, rather than the original exception being propagated back out. The path down for the put was HTable.put->doPut->flushCommits->HConnectionMananger.processBatchOfPuts, which does the retries. The end result is it takes minutes to do a commit because we are doing this checking and retrying, rather than failing out as the CP docs would imply. I feel like this could have serious impacts outside of this patch too.. Unless I'm missing something?
          Hide
          Ted Yu added a comment -

          In processBatchCallback():

                            !(results[i] instanceof DoNotRetryIOException))) {
                      retry = true;
          

          So in your example, you can wrap NumberFormatException in DoNotRetryIOException so that the retries are skipped.

          Show
          Ted Yu added a comment - In processBatchCallback(): !(results[i] instanceof DoNotRetryIOException))) { retry = true ; So in your example, you can wrap NumberFormatException in DoNotRetryIOException so that the retries are skipped.
          Hide
          Jesse Yates added a comment -

          oh... yeah, that would work. First time digging into the code and missed that. Thanks Ted!

          Show
          Jesse Yates added a comment - oh... yeah, that would work. First time digging into the code and missed that. Thanks Ted!
          Hide
          Andrew Purtell added a comment -

          @Jesse: Can we change the title of this issue to simply "Constraints" ?

          Show
          Andrew Purtell added a comment - @Jesse: Can we change the title of this issue to simply "Constraints" ?
          Jesse Yates made changes -
          Summary Add constraints as a top-level feature Constraints
          Hide
          Gary Helmling added a comment -

          Summarizing some comments from a related thread in the dev@hbase.apache.org mailing list:

          http://search-hadoop.com/m/oGvg82YEzA82

          Adding special configuration hooks to HTableDescriptor, say HTableDescriptor.addConstraint(Class c), is likely to lead to long-term clutter if we follow this strategy for each special purpose coprocessor implementation that comes up.

          So instead, I'd suggest we invert this and make the coprocessor implementation responsible for setting it's configuration as attributes on HTableDescriptor. This would follow a pattern something like the following:

          HTableDescriptor htd = new HTableDescriptor(...);
          Constraints.add(htd, MyConstraintImpl.class);
          HBaseAdmin admin = new HBaseAdmin();
          admin.createTable(htd);
          

          The important part being that the Constraints class handles marshalling it's required configuration as attributes on HTableDescriptor, and, in addition, is free to define those configuration helper methods however best makes sense for it's own needs. For the example above, I'm guessing this would be something like:

          public static void add(HTableDescriptor htd, Class<? extends Constraint>... clz)
          

          This keeps HTableDescriptor handling just the common needs (through the arbitrary key/value attributes), while keeping the specific configuration logic with each coprocessor implementation.

          There is an additional challenge, however, in how we can cleanly incorporate access to set the configurations into the HBase shell. Ideally, you'd be able to do something like (exact syntax aside):

          hbase> alter 'mytable', METHOD => 'Constraints.add', VALUE => com.my.MyConstraint
          

          or maybe

          hbase> alter 'mytable', TYPE => 'Constraints', METHOD => 'add', VALUE => com.my.MyConstraint
          

          I think the key problem is figuring out how the shell actually knows about the Constraints class. Some options are:

          1. use the full class name, e.g. "org.apache.hadoop.hbase.coprocessor.Constraints". This is clunky but probably simplest to do.
          2. allowed the Constraints class to register itself via some SPI interface that the shell code can examine for possible providers. This would probably still need the class to be statically initialized somehow.
          3. provide a extensions dir for the shell:
            • extensions drop a simple jruby scriptlet in a file in the dir (say lib/ruby/hbase/ext)
            • the scriptlet does some simple registration of the available methods/commands
            • the shell code loads all files

          There may be better options as well, but hopefully this can serve as the basis for some discussion.

          Show
          Gary Helmling added a comment - Summarizing some comments from a related thread in the dev@hbase.apache.org mailing list: http://search-hadoop.com/m/oGvg82YEzA82 Adding special configuration hooks to HTableDescriptor , say HTableDescriptor.addConstraint(Class c) , is likely to lead to long-term clutter if we follow this strategy for each special purpose coprocessor implementation that comes up. So instead, I'd suggest we invert this and make the coprocessor implementation responsible for setting it's configuration as attributes on HTableDescriptor . This would follow a pattern something like the following: HTableDescriptor htd = new HTableDescriptor(...); Constraints.add(htd, MyConstraintImpl.class); HBaseAdmin admin = new HBaseAdmin(); admin.createTable(htd); The important part being that the Constraints class handles marshalling it's required configuration as attributes on HTableDescriptor , and, in addition, is free to define those configuration helper methods however best makes sense for it's own needs. For the example above, I'm guessing this would be something like: public static void add(HTableDescriptor htd, Class <? extends Constraint>... clz) This keeps HTableDescriptor handling just the common needs (through the arbitrary key/value attributes), while keeping the specific configuration logic with each coprocessor implementation. There is an additional challenge, however, in how we can cleanly incorporate access to set the configurations into the HBase shell. Ideally, you'd be able to do something like (exact syntax aside): hbase> alter 'mytable', METHOD => 'Constraints.add', VALUE => com.my.MyConstraint or maybe hbase> alter 'mytable', TYPE => 'Constraints', METHOD => 'add', VALUE => com.my.MyConstraint I think the key problem is figuring out how the shell actually knows about the Constraints class. Some options are: use the full class name, e.g. "org.apache.hadoop.hbase.coprocessor.Constraints". This is clunky but probably simplest to do. allowed the Constraints class to register itself via some SPI interface that the shell code can examine for possible providers. This would probably still need the class to be statically initialized somehow. provide a extensions dir for the shell: extensions drop a simple jruby scriptlet in a file in the dir (say lib/ruby/hbase/ext) the scriptlet does some simple registration of the available methods/commands the shell code loads all files There may be better options as well, but hopefully this can serve as the basis for some discussion.
          Hide
          Jesse Yates added a comment -

          Initial stab at implementing constraints as a coprocessor to processor to process all the constraints passed to a table.

          Includes:

          • Adding a varargs of constraints to a table
          • Adding a varags of constraint and their own configuration to a table
          • just enabling Constraints on a table (would use any constraints currently stored in the htd)
          • skeleton for disabling constraints (debated feature, since also requires adding a removeCoprocessor feature to htd)
          • integration and unit tests.

          This is pretty close to what I was envisioning, it just needs a couple finishing touches (I think).

          What does everyone think of the impl?

          Show
          Jesse Yates added a comment - Initial stab at implementing constraints as a coprocessor to processor to process all the constraints passed to a table. Includes: Adding a varargs of constraints to a table Adding a varags of constraint and their own configuration to a table just enabling Constraints on a table (would use any constraints currently stored in the htd) skeleton for disabling constraints (debated feature, since also requires adding a removeCoprocessor feature to htd) integration and unit tests. This is pretty close to what I was envisioning, it just needs a couple finishing touches (I think). What does everyone think of the impl?
          Jesse Yates made changes -
          Attachment constraint_as_cp.txt [ 12500962 ]
          Hide
          Ted Yu added a comment -

          All the new files miss license.

          Show
          Ted Yu added a comment - All the new files miss license.
          Hide
          Ted Yu added a comment -

          @Jesse:
          Can you publish your patch on review board ?
          I see some nits which can be better expressed there.

          Show
          Ted Yu added a comment - @Jesse: Can you publish your patch on review board ? I see some nits which can be better expressed there.
          Hide
          Jesse Yates added a comment -

          @Ted: its just supposed to be general design review, hence the lack of licenses (the "finishing touches" mentioned above). I can throw it up on RB though.

          Show
          Jesse Yates added a comment - @Ted: its just supposed to be general design review, hence the lack of licenses (the "finishing touches" mentioned above). I can throw it up on RB though.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          Review request for hbase.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs


          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2870
          -----------------------------------------------------------

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6424>

          This condition can be expressed as !constraints.isEmpty()

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6423>

          Please use curly braces for line 60.
          This log can be combined with log on line 63.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6425>

          This has been done, right ?

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6426>

          Contents of put should be included.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment6427>

          We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled.

          I think the second approach may be better in our case.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment6428>

          Should be 'desc HTableDescriptor to read from'

          • Ted

          On 2011-10-26 22:34:35, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-26 22:34:35)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2870 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6424 > This condition can be expressed as !constraints.isEmpty() src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6423 > Please use curly braces for line 60. This log can be combined with log on line 63. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6425 > This has been done, right ? src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6426 > Contents of put should be included. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment6427 > We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled. I think the second approach may be better in our case. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment6428 > Should be 'desc HTableDescriptor to read from' Ted On 2011-10-26 22:34:35, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-26 22:34:35) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          Also, this patch (clearly) does not include changes to the shell or book documentation. Any pointers on where to look for the former would be really helpful, as I haven't looked into the shell much before (and don't have a ton of ruby experience). However, going to be looking into that next week.

          Show
          Jesse Yates added a comment - Also, this patch (clearly) does not include changes to the shell or book documentation. Any pointers on where to look for the former would be really helpful, as I haven't looked into the shell much before (and don't have a ton of ruby experience). However, going to be looking into that next week.
          Hide
          Ted Yu added a comment -

          See HBASE-4554 which can be used as example for setting coprocessor table attributes.

          Show
          Ted Yu added a comment - See HBASE-4554 which can be used as example for setting coprocessor table attributes.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2917
          -----------------------------------------------------------

          Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:

          1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?
          2) Should we ship a base set of Constraint implementations for common cases?

          Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment6534>

          I think this sentence needs updating to reference Constraints/ConstraintProcessor.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment6535>

          Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment6536>

          Should reference Constraint since BaseConstraint is just a convenience class.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment6521>

          Should we define a base exception here that is thrown? I'm thinking something like:

          ConstraintViolationException extends DoNotRetryIOException

          This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6522>

          If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment6525>

          I think this and the other addConstraints() method could just be named add().

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment6523>

          I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead?

          Also, I would just name this enable().

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment6524>

          In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup.

          By the way, since the name of the class is Constraints, I think this method could just be named disable() – the additional "Constraints" is redundant.

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
          <https://reviews.apache.org/r/2579/#comment6530>

          Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...

          If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?

          It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
          <https://reviews.apache.org/r/2579/#comment6532>

          This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String?

          • Gary

          On 2011-10-26 22:34:35, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-26 22:34:35)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2917 ----------------------------------------------------------- Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions: 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to? 2) Should we ship a base set of Constraint implementations for common cases? Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment6534 > I think this sentence needs updating to reference Constraints/ConstraintProcessor. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment6535 > Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment6536 > Should reference Constraint since BaseConstraint is just a convenience class. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment6521 > Should we define a base exception here that is thrown? I'm thinking something like: ConstraintViolationException extends DoNotRetryIOException This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6522 > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment6525 > I think this and the other addConstraints() method could just be named add(). src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment6523 > I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead? Also, I would just name this enable(). src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment6524 > In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup. By the way, since the name of the class is Constraints, I think this method could just be named disable() – the additional "Constraints" is redundant. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java < https://reviews.apache.org/r/2579/#comment6530 > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/... If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general? It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java < https://reviews.apache.org/r/2579/#comment6532 > This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String? Gary On 2011-10-26 22:34:35, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-26 22:34:35) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-26 23:01:06, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 49

          > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line49>

          >

          > This condition can be expressed as !constraints.isEmpty()

          I was concerned with speed here - maybe a premature optimization and bit of assumption on my part that the check of is empty takes longer than a boolean, and that my way has minimal memory overhead

          On 2011-10-26 23:01:06, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 77

          > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line77>

          >

          > This has been done, right ?

          yup, just some cruft.

          On 2011-10-26 23:01:06, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 83

          > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line83>

          >

          > Contents of put should be included.

          +1

          On 2011-10-26 23:01:06, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 135

          > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line135>

          >

          > We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled.

          >

          > I think the second approach may be better in our case.

          see comments on Gary's comments.

          On 2011-10-26 23:01:06, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 140

          > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line140>

          >

          > Should be 'desc HTableDescriptor to read from'

          +1

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2870
          -----------------------------------------------------------

          On 2011-10-26 22:34:35, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-26 22:34:35)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-26 23:01:06, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 49 > < https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line49 > > > This condition can be expressed as !constraints.isEmpty() I was concerned with speed here - maybe a premature optimization and bit of assumption on my part that the check of is empty takes longer than a boolean, and that my way has minimal memory overhead On 2011-10-26 23:01:06, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 77 > < https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line77 > > > This has been done, right ? yup, just some cruft. On 2011-10-26 23:01:06, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 83 > < https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line83 > > > Contents of put should be included. +1 On 2011-10-26 23:01:06, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 135 > < https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line135 > > > We can either add removeCoprocessor() to HTD or, persist a flag to tell ConstraintProcessor that it is disabled. > > I think the second approach may be better in our case. see comments on Gary's comments. On 2011-10-26 23:01:06, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 140 > < https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line140 > > > Should be 'desc HTableDescriptor to read from' +1 Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2870 ----------------------------------------------------------- On 2011-10-26 22:34:35, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-26 22:34:35) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:

          >

          > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?

          > 2) Should we ship a base set of Constraint implementations for common cases?

          >

          > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.

          1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq.

          So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 12

          > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line12>

          >

          > Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved.

          We can add order preserving, if that makes sense. I can see it for cases where you want to check the cheap constraints first and then do the expensive only if the other ones pass.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39

          > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39>

          >

          > Should we define a base exception here that is thrown? I'm thinking something like:

          >

          > ConstraintViolationException extends DoNotRetryIOException

          >

          > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.

          I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80

          > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80>

          >

          > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.

          I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 75

          > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line75>

          >

          > I think this and the other addConstraints() method could just be named add().

          +1

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 124

          > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line124>

          >

          > I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead?

          >

          > Also, I would just name this enable().

          +1 That was some leftover cruft from development.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 134

          > <https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line134>

          >

          > In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup.

          >

          > By the way, since the name of the class is Constraints, I think this method could just be named disable() – the additional "Constraints" is redundant.

          Just trying to be explicit with the naming.

          I like the idea of the remove, but we just need to make sure it gets well documented as to what happens - I could see a people being confused.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13

          > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13>

          >

          > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...

          >

          > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?

          >

          > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.

          Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 23

          > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line23>

          >

          > This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String?

          Yeah, maybe. This is just an example. Depends on what people want to do...Maybe the fact that the impl can vary so widely makes the case for this to be a test class.

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2917
          -----------------------------------------------------------

          On 2011-10-26 22:34:35, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-26 22:34:35)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-29 02:11:25, Gary Helmling wrote: > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions: > > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to? > 2) Should we ship a base set of Constraint implementations for common cases? > > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here. 1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq. So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 12 > < https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line12 > > > Is this true? The HTD values are stored in a HashMap, so ordering wouldn't be preserved. We can add order preserving, if that makes sense. I can see it for cases where you want to check the cheap constraints first and then do the expensive only if the other ones pass. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39 > < https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39 > > > Should we define a base exception here that is thrown? I'm thinking something like: > > ConstraintViolationException extends DoNotRetryIOException > > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do. I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80 > < https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80 > > > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost. I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 75 > < https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line75 > > > I think this and the other addConstraints() method could just be named add(). +1 On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 124 > < https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line124 > > > I get the idea of being able to enable/disable constraints on demand, without removing the individual constraints configured. But why do we need this and the private addConstraintProcessor() method? Why not move the addConstraintProcessor() implementation here and make addConstraints() call this instead? > > Also, I would just name this enable(). +1 That was some leftover cruft from development. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 134 > < https://reviews.apache.org/r/2579/diff/1/?file=53675#file53675line134 > > > In addition to being able to disable constraint checking by removing the ConstraintProcessor CP, we probably also need a removeConstraints(HTableDescriptor) method that removes the ConstraintProcessor CP and all the "constraint $" attributes on HTD for complete cleanup. > > By the way, since the name of the class is Constraints, I think this method could just be named disable() – the additional "Constraints" is redundant. Just trying to be explicit with the naming. I like the idea of the remove, but we just need to make sure it gets well documented as to what happens - I could see a people being confused. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13 > < https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13 > > > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/... > > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general? > > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values. Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 23 > < https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line23 > > > This doesn't seem right. Wouldn't most ints be stored as Bytes.toBytes(int), not as a String? Yeah, maybe. This is just an example. Depends on what people want to do...Maybe the fact that the impl can vary so widely makes the case for this to be a test class. Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2917 ----------------------------------------------------------- On 2011-10-26 22:34:35, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-26 22:34:35) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc.

          Show
          Ted Yu added a comment - To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc.
          Hide
          Ted Yu added a comment -

          To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry:
          "hbase.constraint.enabled" -> true/false

          This way the ordinal for the constraints would be preserved after arbitrary rounds of enabling/disabling.

          Show
          Ted Yu added a comment - To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry: "hbase.constraint.enabled" -> true/false This way the ordinal for the constraints would be preserved after arbitrary rounds of enabling/disabling.
          Hide
          Jesse Yates added a comment -

          To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc.

          Exactly what I was planning on doing and where I got the inspiration

          To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry:
          "hbase.constraint.enabled" -> true/false

          I was thinking I would just remove the constraint processor from the list of coprocessors. This also gives you complete control over runtime slowdown - if you want to enable constraints, you pay a price, but if not it shouldn't hurt the system speed at all. This would be a good idiom for later, CP based features to follow.
          If we have the true/false it would require doing checking on the regionserver to see if constraints are enabled AND that the CP is loaded to have the same potential, which leads to a lot of code bloat.

          Show
          Jesse Yates added a comment - To preserve order of constraints, we can embed their ordinal in HTD key prefix, such as "constraint 1 $", "constraint 2 $", etc. Exactly what I was planning on doing and where I got the inspiration To enable/disable a constraint, the value (a Configuration object) for the key can carry a special entry: "hbase.constraint.enabled" -> true/false I was thinking I would just remove the constraint processor from the list of coprocessors. This also gives you complete control over runtime slowdown - if you want to enable constraints, you pay a price, but if not it shouldn't hurt the system speed at all. This would be a good idiom for later, CP based features to follow. If we have the true/false it would require doing checking on the regionserver to see if constraints are enabled AND that the CP is loaded to have the same potential, which leads to a lot of code bloat.
          Hide
          Ted Yu added a comment -

          w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1):
          1. retrieve Configuration object from HTD.values map for given Constraint
          2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled"

          There would be impact on performance if the number of constraints is high.

          Show
          Ted Yu added a comment - w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1): 1. retrieve Configuration object from HTD.values map for given Constraint 2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled" There would be impact on performance if the number of constraints is high.
          Hide
          Ted Yu added a comment -

          Should we add method for enabling/disabling individual constraints to Constraints.java ?

            public static void enableConstraint(HTableDescriptor desc, Class<? extends Constraint> clazz) {
          ...
            public static void disableConstraint(HTableDescriptor desc, Class<? extends Constraint> clazz) {
          

          Using the above form, we cannot simply remove the constraint processor.
          We shouldn't remove the constraint corresponding to clazz either because the constraint may have unique Configuration object associated with it.

          Show
          Ted Yu added a comment - Should we add method for enabling/disabling individual constraints to Constraints.java ? public static void enableConstraint(HTableDescriptor desc, Class <? extends Constraint> clazz) { ... public static void disableConstraint(HTableDescriptor desc, Class <? extends Constraint> clazz) { Using the above form, we cannot simply remove the constraint processor. We shouldn't remove the constraint corresponding to clazz either because the constraint may have unique Configuration object associated with it.
          Hide
          Jesse Yates added a comment -

          w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1):
          1. retrieve Configuration object from HTD.values map for given Constraint
          2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled"

          There would be impact on performance if the number of constraints is high.

          I was thinking you meant just using a boolean to enable/disable the entire constraint mechanism, rather than an individual constraint. It is faster to remove the ConstraintProcessor from the CP list as you save the method calls into to ConstraintProcessor to check the enabled/disabled boolean.

          Should we add method for enabling/disabling individual constraints to Constraints.java ?

          This starts to be a bit painful if we want to actually do enforcement of order of constraint application you have to traverse the entire map to check for values (however, if we assume that the map isn't big its not that big of a deal). However, I think, in general, that it is a good idea.

          I think we should also have a remove method as well as the disable method for individual constraints - this is more of a usage consideration rather than a runtime one. Say you added a constraint that was wrong - removing it rather than disabling it is definitely necessary.

          Also, we should also consider having:

          public static void setConfiguration(HTableDescriptor desc, Class<? extends Constraint> clazz, Configuration conf){}
          

          That overrides the configuration set for a constraint.

          Show
          Jesse Yates added a comment - w.r.t. performance after several rounds of enabling/disabling, the following operations are both O(1): 1. retrieve Configuration object from HTD.values map for given Constraint 2. retrieve the boolean in this Configuration object for "hbase.constraint.enabled" There would be impact on performance if the number of constraints is high. I was thinking you meant just using a boolean to enable/disable the entire constraint mechanism, rather than an individual constraint. It is faster to remove the ConstraintProcessor from the CP list as you save the method calls into to ConstraintProcessor to check the enabled/disabled boolean. Should we add method for enabling/disabling individual constraints to Constraints.java ? This starts to be a bit painful if we want to actually do enforcement of order of constraint application you have to traverse the entire map to check for values (however, if we assume that the map isn't big its not that big of a deal). However, I think, in general, that it is a good idea. I think we should also have a remove method as well as the disable method for individual constraints - this is more of a usage consideration rather than a runtime one. Say you added a constraint that was wrong - removing it rather than disabling it is definitely necessary. Also, we should also consider having: public static void setConfiguration(HTableDescriptor desc, Class <? extends Constraint> clazz, Configuration conf){} That overrides the configuration set for a constraint.
          Hide
          Ted Yu added a comment -

          @Jesse:
          You can decide what to put in this JIRA as long as future enhancement can be easily accommodated.

          Show
          Ted Yu added a comment - @Jesse: You can decide what to put in this JIRA as long as future enhancement can be easily accommodated.
          Hide
          Jesse Yates added a comment -

          @Ted: ok, fair enough. I'm just trying to work through the possibilities and get feedback (thanks btw!).

          Show
          Jesse Yates added a comment - @Ted: ok, fair enough. I'm just trying to work through the possibilities and get feedback (thanks btw!).
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-10-31 00:48:49.738577)

          Review request for hbase.

          Changes
          -------

          Updated based on comments on the last patch and in HBASE-4605 (corresponding JIRA).
          Added feautures:

          • enable
          • disable
          • remove
          • enable constraint
          • disable constraint
          • remove constraint
          • actually enforcing ordering of constraints based on add
          • update constraint
          • removal of CPs in HTD
            Also, updated tests to cover new cases (which all pass locally).

          Still TODO: tie constraints into shell

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-31 00:48:49.738577) Review request for hbase. Changes ------- Updated based on comments on the last patch and in HBASE-4605 (corresponding JIRA). Added feautures: enable disable remove enable constraint disable constraint remove constraint actually enforcing ordering of constraints based on add update constraint removal of CPs in HTD Also, updated tests to cover new cases (which all pass locally). Still TODO: tie constraints into shell Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2934
          -----------------------------------------------------------

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment6568>

          Whoops - missed that one.

          • Jesse

          On 2011-10-31 00:48:49, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-31 00:48:49)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2934 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment6568 > Whoops - missed that one. Jesse On 2011-10-31 00:48:49, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-31 00:48:49) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          In the above patch, why do we need two loops in Constraints.remove(HTableDescriptor) ?
          Can we call desc.remove(e.getKey()) in the first loop ?

          Show
          Ted Yu added a comment - In the above patch, why do we need two loops in Constraints.remove(HTableDescriptor) ? Can we call desc.remove(e.getKey()) in the first loop ?
          Hide
          Ted Yu added a comment -

          I think the name of table should be added to the following exception:

                throw new IllegalArgumentException("Constraint: " + clazz.getName()
                    + " is not associated with this table.");
          

          Constraints.getKeyValueForClass() may return null. I think we should check for null in remove(HTableDescriptor, Class):

               desc.remove(e.getKey().get());
          

          and other methods.

          Show
          Ted Yu added a comment - I think the name of table should be added to the following exception: throw new IllegalArgumentException( "Constraint: " + clazz.getName() + " is not associated with this table." ); Constraints.getKeyValueForClass() may return null. I think we should check for null in remove(HTableDescriptor, Class): desc.remove(e.getKey().get()); and other methods.
          Hide
          Jesse Yates added a comment -

          @Ted: concurrent modification exception - an inherent problem of using the iterator on the hash map (actually tried that first though )

          Show
          Jesse Yates added a comment - @Ted: concurrent modification exception - an inherent problem of using the iterator on the hash map (actually tried that first though )
          Hide
          Ted Yu added a comment -

          @Jesse:
          I only see synchronized keyword on Constraints.add().
          Have you tried using synchronization on other methods ?

          Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described.

          If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism.
          The current approach deals with serialization itself. This is not as flexible as the above approach.

          Show
          Ted Yu added a comment - @Jesse: I only see synchronized keyword on Constraints.add(). Have you tried using synchronization on other methods ? Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described. If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism. The current approach deals with serialization itself. This is not as flexible as the above approach.
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions:

          >

          > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to?

          > 2) Should we ship a base set of Constraint implementations for common cases?

          >

          > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here.

          Jesse Yates wrote:

          1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq.

          So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution.

          Okay, seems fine to me. Just wanted to raise the questions.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80

          > <https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80>

          >

          > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost.

          Jesse Yates wrote:

          I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else.

          Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors – by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13

          > <https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13>

          >

          > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/...

          >

          > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general?

          >

          > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values.

          Jesse Yates wrote:

          Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones

          Maybe it should just go in the documentation then? Either in package level javadoc or a section of the HBase book? Maybe longer term we need a separate "examples" maven module for this and other extensions points. But the current implementation doesn't seem useful out of the box, since it forces integers to be stored as strings... in order to check that they're integers. Seems odd.

          On 2011-10-29 02:11:25, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39

          > <https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39>

          >

          > Should we define a base exception here that is thrown? I'm thinking something like:

          >

          > ConstraintViolationException extends DoNotRetryIOException

          >

          > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do.

          Jesse Yates wrote:

          I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy.

          I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review2917
          -----------------------------------------------------------

          On 2011-10-31 00:48:49, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-10-31 00:48:49)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-10-29 02:11:25, Gary Helmling wrote: > Jessie, this looks like a pretty good start. In addition to the individual comments, a couple of general questions: > > 1) Should ConstraintProcessor support some standard way of mapping Constraint implementations to the families/qualifiers they should apply to? > 2) Should we ship a base set of Constraint implementations for common cases? > > Even if the answer to both is "yes", they could be addressed as follow-up JIRAs. But it would be good to think through the end goal here. Jesse Yates wrote: 1) I was thinking an individual constriant could check any/all of the cf/cq being used. We can do it via (2) where we have an abstract Constraint that makes it easy to check a given cf/cq. So, no (kinda) and yes. For (2), I think it will be community usage/findings as to what people find useful - we can decide on a case-by-case basis if it is a valid contribution. Okay, seems fine to me. Just wanted to raise the questions. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 80 > < https://reviews.apache.org/r/2579/diff/1/?file=53674#file53674line80 > > > If we use a defined exception class for constraint violations (see comment above), I think we can omit this try/catch block. Expected exceptions would already subclass DoNotRetryIOException, so no need to wrap it here. Other (non-expected) Throwables would be handled higher up by RegionCoprocessorHost. Jesse Yates wrote: I don't think that runtime exceptions on a constraint should be allowed to propagate up the CPHost - if the constraint fails then that put should fail, but not anything else. Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors – by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 13 > < https://reviews.apache.org/r/2579/diff/1/?file=53676#file53676line13 > > > Is this intended as a test class or as a usable implementation? If a test class, should be under src/test/java/... > > If a usable implementation, then, looking at it, I wonder how useful it is without a way of mapping constraints to individual qualifiers. Do you think that's something ConstraintProcessor should provide in general? > > It could be left up to end user implementations of Constraint to match which KeyValues they should apply to. But then I don't think any implementations we can bundle will have much value, unless they provide for their own qualifier matching via configuration values. Jesse Yates wrote: Its meant to be both an example and a test util. I can move it into test, but I can see cases where people might want to just check to make sure they only store values. Yes, it a really simple use case, but it also helps people to build more complex ones Maybe it should just go in the documentation then? Either in package level javadoc or a section of the HBase book? Maybe longer term we need a separate "examples" maven module for this and other extensions points. But the current implementation doesn't seem useful out of the box, since it forces integers to be stored as strings... in order to check that they're integers. Seems odd. On 2011-10-29 02:11:25, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 39 > < https://reviews.apache.org/r/2579/diff/1/?file=53673#file53673line39 > > > Should we define a base exception here that is thrown? I'm thinking something like: > > ConstraintViolationException extends DoNotRetryIOException > > This way we don't need to wrap in DoNotRetryIOException later and we give implementers a bit more guidance on what to do. Jesse Yates wrote: I wanted to let people throw whatever exception they wanted to so they can be specific as to what happened with minimal overhead. For instance, the IntegerConstraint lets the constraint just check to and then throw a NumberFormatException if it fails. However, the code for the user (the actual constraint implementation) is super concise and easy. I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review2917 ----------------------------------------------------------- On 2011-10-31 00:48:49, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-10-31 00:48:49) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          @Ted:

          I only see synchronized keyword on Constraints.add().
          Have you tried using synchronization on other methods ?

          No I haven't, but this was the beginnings of making the setting synchronized. However, after thinking about it, that doesn't really make all that much sense for a bunch of static setting methods. I was going to push this off onto the user.

          Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described.

          Seems a little excessive especially since all the synchronization on HTD has been done by users when necessary in the past. My hunch is that this is rarely an issue.

          If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism.
          The current approach deals with serialization itself. This is not as flexible as the above approach.

          +1 Making the changes. My initial thought was to mimic how CPs are added to the table, which handle writing their own bytes. But, yeah your recommendation would be makes it much easier (and consistent).

          @Gary:

          Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors – by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers.

          AND

          I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden.

          +1 having the user propagate a DoNoRetryException subclass (eg. ConstraintException) - you make a good point. Working on the best way to handle unload/total failure.

          Sometimes people do weird things (I've seen people doing exactly what I was testing against), so while ugly, is not unheard of. An examples module would be great - I'll comment on 4336 about that. In the mean time, maybe just having an examples package (main/java/…hbase.constraint.examples) that can be updated as people find more uses for constraints? Though I'm not adverse to the book either.

          Show
          Jesse Yates added a comment - @Ted: I only see synchronized keyword on Constraints.add(). Have you tried using synchronization on other methods ? No I haven't, but this was the beginnings of making the setting synchronized. However, after thinking about it, that doesn't really make all that much sense for a bunch of static setting methods. I was going to push this off onto the user. Also, HTableDescriptor.values is protected field. We can change its actual implementation to ConcurrentHashMap, etc to accommodate for the concurrency you described. Seems a little excessive especially since all the synchronization on HTD has been done by users when necessary in the past. My hunch is that this is rarely an issue. If we store metadata about constraints in the Configuration object as I described @ 29/Oct/11 04:20, we utilize the available serialization mechanism. The current approach deals with serialization itself. This is not as flexible as the above approach. +1 Making the changes. My initial thought was to mimic how CPs are added to the table, which handle writing their own bytes. But, yeah your recommendation would be makes it much easier (and consistent). @Gary: Some runtime exceptions could indicate programming errors or bugs, in which case I think it's best to handle them the same way we handle unexpected errors in coprocessors – by either unloading or aborting, depending on configuration. We need to be careful for what we allow from user code running in process on region servers. AND I disagree here. Treating all possible exceptions the same seems wrong. An OutOfMemoryError is not equivalent to a NumberFormatException. We need some structure in place for what types of errors are allowed/expected, instead of swallowing everything and assuming it's just a constraint violation. I don't think asking the Constraint implementations to think about what errors they want to throw is much of a burden. +1 having the user propagate a DoNoRetryException subclass (eg. ConstraintException) - you make a good point. Working on the best way to handle unload/total failure. Sometimes people do weird things (I've seen people doing exactly what I was testing against), so while ugly, is not unheard of. An examples module would be great - I'll comment on 4336 about that. In the mean time, maybe just having an examples package (main/java/…hbase.constraint.examples) that can be updated as people find more uses for constraints? Though I'm not adverse to the book either.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-11-17 07:53:56.106789)

          Review request for hbase.

          Changes
          -------

          Based on Ted's suggestion of using the inherent serialization in HTD, I attempted to switch over. However, this was met with some problems of going to/from String/byte[] in the value. This lead to the large number of changes in the current patch.

          Essentially, I abstracted out the serialization/deserialization mechanism from the methods into two sets of guava functions (one set for keys, one for values) so we can encapsulate the serialization and in the future make it easier to change. With this encapsulation, I also updated the way CPs are stored to also use the built-in HTD serialization/deserialization mechanisms.

          This patch also attempts to fix the remaining issues recently brought up in 4605, namely:

          • Using a ConstraintException to specify an allowed constraint
          • Just unloading the entire ConstraintProcessor on a runtimeexception
          • dropping all the attempts at making Constraints (the class) thread safe

          I'm working on the shell stuff (been busy), but figured I would throw this up in the meantime so we can get the java stuff solid.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java ef47d0d
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-17 07:53:56.106789) Review request for hbase. Changes ------- Based on Ted's suggestion of using the inherent serialization in HTD, I attempted to switch over. However, this was met with some problems of going to/from String/byte[] in the value. This lead to the large number of changes in the current patch. Essentially, I abstracted out the serialization/deserialization mechanism from the methods into two sets of guava functions (one set for keys, one for values) so we can encapsulate the serialization and in the future make it easier to change. With this encapsulation, I also updated the way CPs are stored to also use the built-in HTD serialization/deserialization mechanisms. This patch also attempts to fix the remaining issues recently brought up in 4605, namely: Using a ConstraintException to specify an allowed constraint Just unloading the entire ConstraintProcessor on a runtimeexception dropping all the attempts at making Constraints (the class) thread safe I'm working on the shell stuff (been busy), but figured I would throw this up in the meantime so we can get the java stuff solid. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java ef47d0d src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          I got compilation error trying to run integration test:

          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile (default-testCompile) on project hbase: Compilation failure: Compilation failure:
          [ERROR] /Users/zhihyu/trunk-hbase/src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java:[107,26] cannot find symbol
          [ERROR] symbol  : class AllFailConstraint
          
          Show
          Ted Yu added a comment - I got compilation error trying to run integration test: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile ( default -testCompile) on project hbase: Compilation failure: Compilation failure: [ERROR] /Users/zhihyu/trunk-hbase/src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java:[107,26] cannot find symbol [ERROR] symbol : class AllFailConstraint
          Hide
          Ted Yu added a comment -

          I went through HTableDescriptor where the serialization/deserialization logic is defined.

          SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings).
          Same with DESERIALIZE_KEY and DESERIALIZE_VALUE.

          getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass().

          Show
          Ted Yu added a comment - I went through HTableDescriptor where the serialization/deserialization logic is defined. SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings). Same with DESERIALIZE_KEY and DESERIALIZE_VALUE. getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass().
          Hide
          Jesse Yates added a comment -

          I got compilation error trying to run integration test:

          Hmmm, something was a little wonky - had test/ excluded to ignore some build issues, but that neglected all the tests I added. But FWIW it passes locally Throwing up the correct version momentarily

          Also, please ignore package-info.java - its on the TODO list and just using a placeholder for the moment.

          SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings).
          Same with DESERIALIZE_KEY and DESERIALIZE_VALUE.

          I was thinking it would be better to split them into two different elements since we may want to change how we do the serialization for each in the future, and having the independence makes it much easier to swap and adds negligible overhead.

          getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass().

          +1 thought it might be useful later, but we can always add it as needed. Making the fix in the latest patch.

          Show
          Jesse Yates added a comment - I got compilation error trying to run integration test: Hmmm, something was a little wonky - had test/ excluded to ignore some build issues, but that neglected all the tests I added. But FWIW it passes locally Throwing up the correct version momentarily Also, please ignore package-info.java - its on the TODO list and just using a placeholder for the moment. SERIALIZE_KEY and SERIALIZE_VALUE perform the same transformation. I think they should be combined into one Function, possibly named serializer so that naming for Functions is consistent (with transformToStrings). Same with DESERIALIZE_KEY and DESERIALIZE_VALUE. I was thinking it would be better to split them into two different elements since we may want to change how we do the serialization for each in the future, and having the independence makes it much easier to swap and adds negligible overhead. getValueAsBytes() is only used once. I think we can just inline what it does in getKeyValueForClass(). +1 thought it might be useful later, but we can always add it as needed. Making the fix in the latest patch.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-11-17 18:56:42.730664)

          Review request for hbase.

          Changes
          -------

          Adding in test classes that we accidentally dropped in last patch.

          Also, removing HTableDescriptor.getValueAsBytes(String) in favor of on demand, in-line conversion using static calls.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-17 18:56:42.730664) Review request for hbase. Changes ------- Adding in test classes that we accidentally dropped in last patch. Also, removing HTableDescriptor.getValueAsBytes(String) in favor of on demand, in-line conversion using static calls. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3324
          -----------------------------------------------------------

          Quick question: the security coprocessor lives in a separate directory – /security/src/... — should this coprocessor live in its own as well?

          These coprocessors could eventually be put into separate submodules to keep the core thinner.

          This could likely be handled by follow on jira.

          • jmhsieh

          On 2011-11-17 18:56:42, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-17 18:56:42)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3324 ----------------------------------------------------------- Quick question: the security coprocessor lives in a separate directory – /security/src/... — should this coprocessor live in its own as well? These coprocessors could eventually be put into separate submodules to keep the core thinner. This could likely be handled by follow on jira. jmhsieh On 2011-11-17 18:56:42, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-17 18:56:42) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-17 20:51:45, jmhsieh wrote:

          > Quick question: the security coprocessor lives in a separate directory – /security/src/... — should this coprocessor live in its own as well?

          >

          > These coprocessors could eventually be put into separate submodules to keep the core thinner.

          >

          > This could likely be handled by follow on jira.

          I was debating the idea of moving into it's own module. However, that feels a bit heavy weight for what is honestly just a handful of classes.

          There could also be the argument that constraints need to be 'core' piece of a database (at least that is the feeling I get from many ex-SQLers).

          Follow on ticket would be a good place, if we decide its the right move - pulling it out wouldn't be that bad.

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3324
          -----------------------------------------------------------

          On 2011-11-17 18:56:42, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-17 18:56:42)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-17 20:51:45, jmhsieh wrote: > Quick question: the security coprocessor lives in a separate directory – /security/src/... — should this coprocessor live in its own as well? > > These coprocessors could eventually be put into separate submodules to keep the core thinner. > > This could likely be handled by follow on jira. I was debating the idea of moving into it's own module. However, that feels a bit heavy weight for what is honestly just a handful of classes. There could also be the argument that constraints need to be 'core' piece of a database (at least that is the feeling I get from many ex-SQLers). Follow on ticket would be a good place, if we decide its the right move - pulling it out wouldn't be that bad. Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3324 ----------------------------------------------------------- On 2011-11-17 18:56:42, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-17 18:56:42) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          I got the following for patch v5:

          testEnableDisableRemove(org.apache.hadoop.hbase.constraint.TestConstraints)  Time elapsed: 0.005 sec  <<< ERROR!
          java.lang.NullPointerException
            at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:425)
            at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1153)
            at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1150)
            at org.apache.hadoop.hbase.constraint.Constraints.getKeyValueForClass(Constraints.java:151)
            at org.apache.hadoop.hbase.constraint.Constraints.has(Constraints.java:137)
            at org.apache.hadoop.hbase.constraint.TestConstraints.testEnableDisableRemove(TestConstraints.java:121)
          
            testIntegerConstraint(org.apache.hadoop.hbase.constraint.TestIntegerConstraint): Unexpected exception, expected<java.lang.NumberFormatException> but was<org.apache.hadoop.hbase.constraint.ConstraintException>
          
          Show
          Ted Yu added a comment - I got the following for patch v5: testEnableDisableRemove(org.apache.hadoop.hbase.constraint.TestConstraints) Time elapsed: 0.005 sec <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:425) at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1153) at org.apache.hadoop.hbase.HTableDescriptor$3.apply(HTableDescriptor.java:1150) at org.apache.hadoop.hbase.constraint.Constraints.getKeyValueForClass(Constraints.java:151) at org.apache.hadoop.hbase.constraint.Constraints.has(Constraints.java:137) at org.apache.hadoop.hbase.constraint.TestConstraints.testEnableDisableRemove(TestConstraints.java:121) testIntegerConstraint(org.apache.hadoop.hbase.constraint.TestIntegerConstraint): Unexpected exception, expected<java.lang.NumberFormatException> but was<org.apache.hadoop.hbase.constraint.ConstraintException>
          Hide
          Jesse Yates added a comment -

          Sorry about the quality of these patches Ted, something must be messed up with my IDE to not be catching some of this...

          Anyways, yes, your right, that is broken. Comes from the switch to throwing the ConstraintException rather allowing an exception to be thrown.

          Also, it looks like testEnableDisableRemove has gotten messed up from dropping the getValueAsBytes(). Fixing and reposting.

          Show
          Jesse Yates added a comment - Sorry about the quality of these patches Ted, something must be messed up with my IDE to not be catching some of this... Anyways, yes, your right, that is broken. Comes from the switch to throwing the ConstraintException rather allowing an exception to be thrown. Also, it looks like testEnableDisableRemove has gotten messed up from dropping the getValueAsBytes(). Fixing and reposting.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-11-17 22:23:09.015159)

          Review request for hbase.

          Changes
          -------

          Fixed test bugs from last 2 sets of changes (thx Ted!).

          Ran tests: mvn clean test -Dtest=Constraint
          They all pass locally.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-17 22:23:09.015159) Review request for hbase. Changes ------- Fixed test bugs from last 2 sets of changes (thx Ted!). Ran tests: mvn clean test -Dtest= Constraint They all pass locally. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 99875b8 src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          @Jesse:
          Patch v6 doesn't apply cleanly:

          Hunk #13 FAILED at 1135.
          1 out of 13 hunks FAILED -- saving rejects to file src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java.rej
          

          Do you mind uploading a patch (--no-prefix) which applies to TRUNK so that HadoopQA can run through it ?

          Thanks

          Show
          Ted Yu added a comment - @Jesse: Patch v6 doesn't apply cleanly: Hunk #13 FAILED at 1135. 1 out of 13 hunks FAILED -- saving rejects to file src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java.rej Do you mind uploading a patch (--no-prefix) which applies to TRUNK so that HadoopQA can run through it ? Thanks
          Hide
          Jesse Yates added a comment -

          Yeah, sure. I actually just ran into the same issue trying to work on the shell stuff.

          Pushing up new version shortly.

          Show
          Jesse Yates added a comment - Yeah, sure. I actually just ran into the same issue trying to work on the shell stuff. Pushing up new version shortly.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-11-23 21:19:56.263794)

          Review request for hbase.

          Changes
          -------

          Updating to current trunk to take into account changes in HTD and for Hadoop QA. Otherwise, no changes from last diff.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-23 21:19:56.263794) Review request for hbase. Changes ------- Updating to current trunk to take into account changes in HTD and for Hadoop QA. Otherwise, no changes from last diff. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Ted Yu made changes -
          Attachment 4605.v7 [ 12504935 ]
          Hide
          Ted Yu added a comment -

          Patch testing v7.

          Show
          Ted Yu added a comment - Patch testing v7.
          Ted Yu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Jesse Yates added a comment -

          Attaching patch.

          Includes java implementation and documentation.

          Moving shell modification to a follow-on ticket.

          Show
          Jesse Yates added a comment - Attaching patch. Includes java implementation and documentation. Moving shell modification to a follow-on ticket.
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v1.patch [ 12505275 ]
          Hide
          Jesse Yates added a comment -

          oops, wrong file. This should be right.

          Show
          Jesse Yates added a comment - oops, wrong file. This should be right.
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v2.patch [ 12505277 ]
          Hide
          Jesse Yates added a comment -

          Forgot --no-prefix. Let's try this again.

          Show
          Jesse Yates added a comment - Forgot --no-prefix. Let's try this again.
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v3.patch [ 12505280 ]
          Hide
          Hadoop QA added a comment - - edited

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -162 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.TestRollingRestart
          org.apache.hadoop.hbase.util.TestRegionSplitter
          org.apache.hadoop.hbase.io.hfile.TestLruBlockCache

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - - edited -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRollingRestart org.apache.hadoop.hbase.util.TestRegionSplitter org.apache.hadoop.hbase.io.hfile.TestLruBlockCache Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console This message is automatically generated.
          Hide
          Jesse Yates added a comment -

          Hmmm, so it looks like most of them failed. So far, its passing on my local machine (OSX). Could this be a build system overload issue? Or something along those lines?

          Show
          Jesse Yates added a comment - Hmmm, so it looks like most of them failed. So far, its passing on my local machine (OSX). Could this be a build system overload issue? Or something along those lines?
          Hide
          Ted Yu added a comment -

          From https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/org.apache.hadoop.hbase/TestRegionRebalancing/org_apache_hadoop_hbase_TestRegionRebalancing/:

          java.lang.OutOfMemoryError: unable to create new native thread
          

          Report from HadoopQA was not reliable these days.

          Show
          Ted Yu added a comment - From https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/org.apache.hadoop.hbase/TestRegionRebalancing/org_apache_hadoop_hbase_TestRegionRebalancing/: java.lang.OutOfMemoryError: unable to create new native thread Report from HadoopQA was not reliable these days.
          Jesse Yates made changes -
          Link This issue blocks HBASE-4879 [ HBASE-4879 ]
          Hide
          Ted Yu added a comment -

          About the changes to book.xml:

          +    Constraints could be also be used to ...
          

          The above should read 'Constraints could also be used to'

          where that checking enabled
          

          The above should read 'where integrity checking is enabled'

          Show
          Ted Yu added a comment - About the changes to book.xml: + Constraints could be also be used to ... The above should read 'Constraints could also be used to' where that checking enabled The above should read 'where integrity checking is enabled'
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3551
          -----------------------------------------------------------

          Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good.

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2579/#comment7921>

          Nice addition to have

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2579/#comment7913>

          I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment7916>

          style nit: if body should be contained in braces

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment7914>

          Add constraints.size() to this message and you don't need the separate LOG.debug() message.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment7912>

          style nit: for loop body should be enclosed in braces

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment7917>

          I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString()

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment7918>

          Be consistent with braces here

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment7915>

          if / else bodies should be contained in braces.

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
          <https://reviews.apache.org/r/2579/#comment7922>

          This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer.

          I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work.

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment7919>

          This differs from what seems to be the latest patch in JIRA (java_HBASE-4605_v3.txt). Are there other differences than the javadoc?

          For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch.

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
          <https://reviews.apache.org/r/2579/#comment7920>

          brace alignment and indentation

          • Gary

          On 2011-11-23 21:19:56, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-23 21:19:56)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3551 ----------------------------------------------------------- Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good. src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2579/#comment7921 > Nice addition to have src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2579/#comment7913 > I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment7916 > style nit: if body should be contained in braces src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment7914 > Add constraints.size() to this message and you don't need the separate LOG.debug() message. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment7912 > style nit: for loop body should be enclosed in braces src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment7917 > I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString() src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment7918 > Be consistent with braces here src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment7915 > if / else bodies should be contained in braces. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java < https://reviews.apache.org/r/2579/#comment7922 > This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer. I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work. src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment7919 > This differs from what seems to be the latest patch in JIRA (java_ HBASE-4605 _v3.txt). Are there other differences than the javadoc? For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch. src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java < https://reviews.apache.org/r/2579/#comment7920 > brace alignment and indentation Gary On 2011-11-23 21:19:56, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-23 21:19:56) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good.

          Uploading the latest (with some of your recommended changes) after this. Couple of concerns may still need to be worked out around IntegerConstraint and the mods to HTD. Let me know what you think.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 22

          > <https://reviews.apache.org/r/2579/diff/6/?file=59947#file59947line22>

          >

          > This differs from what seems to be the latest patch in JIRA (java_HBASE-4605_v3.txt). Are there other differences than the javadoc?

          >

          > For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch.

          Yeah, this is the old version. Whats up on the ticket is just documentation. I'll put up the latest here after making the changes.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 250

          > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line250>

          >

          > if / else bodies should be contained in braces.

          done.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 233

          > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line233>

          >

          > Be consistent with braces here

          Just following the requirements of java Done.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 22

          > <https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line22>

          >

          > I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString()

          By pulling the functionality out of the HTD we can be sure that outside classes don't have to worry about making sure they use the 'right' (de)serialization of they keys. Right now we don't want to to anything more than just Bytes.toString(), but in the future we could make it something more complex. For example, take the case where user specified keys are prepended with a special identifier, so we can tell the difference between the two. Alternatively, we could do some cool stuff to cut down on object creation (potentially) with the key conversion or make it faster or more compact or w/e.

          Basically, I'm just adding the encapsulation so classes outside of the HTD don't have to know about the fact that the HTD is using Bytes.toString() (and encapsulation is good, right?). This way it is obvious, to an outside class, what is going - oh, it just serializing the key. This makes the addCP code much cleaner too.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 86

          > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line86>

          >

          > style nit: for loop body should be enclosed in braces

          Done

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 76

          > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line76>

          >

          > Add constraints.size() to this message and you don't need the separate LOG.debug() message.

          Done

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 61

          > <https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line61>

          >

          > style nit: if body should be contained in braces

          Done.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 32

          > <https://reviews.apache.org/r/2579/diff/6/?file=59946#file59946line32>

          >

          > This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer.

          >

          > I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work.

          The only problem with using the Bytes deserialization checking is that even if you put a string in as bytes, it can be read out as an integer (in certain cases). Therefore, you could get a case where something that should not be an integer is allowed when it really ought not to be.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java, line 90

          > <https://reviews.apache.org/r/2579/diff/6/?file=59954#file59954line90>

          >

          > brace alignment and indentation

          Done.

          On 2011-11-29 08:36:29, Gary Helmling wrote:

          > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1166

          > <https://reviews.apache.org/r/2579/diff/6/?file=59940#file59940line1166>

          >

          > I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc.

          See response below.

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3551
          -----------------------------------------------------------

          On 2011-11-23 21:19:56, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-23 21:19:56)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-11-29 08:36:29, Gary Helmling wrote: > Most of what I see is just style issues, and a number of the HTableDescriptor changes seem unnecessary. If we clean up those, and assuming the latest patch has some documentation, I think this is looking pretty good. Uploading the latest (with some of your recommended changes) after this. Couple of concerns may still need to be worked out around IntegerConstraint and the mods to HTD. Let me know what you think. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 22 > < https://reviews.apache.org/r/2579/diff/6/?file=59947#file59947line22 > > > This differs from what seems to be the latest patch in JIRA (java_ HBASE-4605 _v3.txt). Are there other differences than the javadoc? > > For a patch of this size, review still belongs here in review board (even with HadoopQA patch testing), so we should keep both versions in sync. Please update the diff here with the latest patch. Yeah, this is the old version. Whats up on the ticket is just documentation. I'll put up the latest here after making the changes. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 250 > < https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line250 > > > if / else bodies should be contained in braces. done. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 233 > < https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line233 > > > Be consistent with braces here Just following the requirements of java Done. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 22 > < https://reviews.apache.org/r/2579/diff/6/?file=59945#file59945line22 > > > I see the usage outside of HTD now, but still don't see where this provides more utility than Bytes.toString() By pulling the functionality out of the HTD we can be sure that outside classes don't have to worry about making sure they use the 'right' (de)serialization of they keys. Right now we don't want to to anything more than just Bytes.toString(), but in the future we could make it something more complex. For example, take the case where user specified keys are prepended with a special identifier, so we can tell the difference between the two. Alternatively, we could do some cool stuff to cut down on object creation (potentially) with the key conversion or make it faster or more compact or w/e. Basically, I'm just adding the encapsulation so classes outside of the HTD don't have to know about the fact that the HTD is using Bytes.toString() (and encapsulation is good, right?). This way it is obvious, to an outside class, what is going - oh, it just serializing the key. This makes the addCP code much cleaner too. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 86 > < https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line86 > > > style nit: for loop body should be enclosed in braces Done On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 76 > < https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line76 > > > Add constraints.size() to this message and you don't need the separate LOG.debug() message. Done On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java, line 61 > < https://reviews.apache.org/r/2579/diff/6/?file=59944#file59944line61 > > > style nit: if body should be contained in braces Done. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java, line 32 > < https://reviews.apache.org/r/2579/diff/6/?file=59946#file59946line32 > > > This still seems odd to me. I can't see storing an integer value encoded as a string, in order to be able to check for it being a valid integer. > > I would think most users would be storing integer values using Bytes.toInt(), for which the checks here would not work. The only problem with using the Bytes deserialization checking is that even if you put a string in as bytes, it can be read out as an integer (in certain cases). Therefore, you could get a case where something that should not be an integer is allowed when it really ought not to be. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java, line 90 > < https://reviews.apache.org/r/2579/diff/6/?file=59954#file59954line90 > > > brace alignment and indentation Done. On 2011-11-29 08:36:29, Gary Helmling wrote: > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1166 > < https://reviews.apache.org/r/2579/diff/6/?file=59940#file59940line1166 > > > I don't see why these changes are necessary when I don't see them being used anywhere outside of HTD? Do these additions buy us anything? In my opinion, they're not making the code any clearer than the current Byte.toString(), etc. See response below. Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3551 ----------------------------------------------------------- On 2011-11-23 21:19:56, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-23 21:19:56) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          Latest patch does not have labeling of tests since I wanted to wait and see what dev@ thinks about the latest testing stuff.

          Show
          Jesse Yates added a comment - Latest patch does not have labeling of tests since I wanted to wait and see what dev@ thinks about the latest testing stuff.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-11-29 20:19:41.250509)

          Review request for hbase.

          Changes
          -------

          Updating to incorporate Gary and Ted's latest (as of 11/29) comments.

          Mostly cleanup in syntax. Also pushing up latest documentation changes that we previously on the Jira.

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/docbkx/book.xml 3c12169
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-29 20:19:41.250509) Review request for hbase. Changes ------- Updating to incorporate Gary and Ted's latest (as of 11/29) comments. Mostly cleanup in syntax. Also pushing up latest documentation changes that we previously on the Jira. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/docbkx/book.xml 3c12169 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3610
          -----------------------------------------------------------

          Main concerns as same as previously – the HTableDescriptor changes and IntegerConstraint:

          1) HTD changes introduce a dependency on Guava. I thought we had eliminated all client-side dependencies on Guava?
          2) I think serializing constraint configurations as XML would be useful, as they would then be human readable and require less care to preserve previous serialization formats on upgrade when the property implementations change.
          3) I'm still not seeing the use of IntegerConstraint as the one Constraint implementation that we ship as part of core (as opposed to it being an example for documentation). Is there a specific use where you're targeting this?

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2579/#comment8047>

          I still don't think this gives us true encapsulation, since the HTD.setValue() allows arbitrary byte[] for values, so there's no way of knowing if (DE)SERIALIZE_KEY was actually used. So is this really better than Bytes.toString()?

          I also thought that we had eliminated Guava as a client side dependency? This would re-introduce it. That seems the bigger concern unless we've agreed to a client-side Guava dependency somewhere.

          Why not handle the values as Strings and serialize the Contraint configuration objects as XML? In this case, I think human readable is actually good.

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          <https://reviews.apache.org/r/2579/#comment8059>

          Omit copyright line

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8058>

          Omit copyright line

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8055>

          Should constraints apply to Increments and Deletes as well? I don't know if they should or not, and even if they do, it doesn't need to be part of this issue. Just something to consider.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
          <https://reviews.apache.org/r/2579/#comment8057>

          Omit copyright line

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment8056>

          Copyright line not necessary in file headers.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8060>

          Having HTD.getValue() convert to String here and then immediately converting back to byte[] is odd. I think either add HTD.getValueAsBytes() returning the raw byte[] or serialize the Configuration as XML as I suggested and just handle it as a String here.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8053>

          The manual serialization of properties here seems likely to be brittle and subject to breakage on upgrade if the implementation changes (unless backwards compatibility is explicitly accounted for).

          This actually seems like a decent case for using the Configuration class' XML serialization support. The output will be much larger, but more resistant to changes in the properties serialized. Just clone the input configuration object, set the "enabled" and "priority" properties under known keys, and serialize to a string using Configuration.writeXML().

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
          <https://reviews.apache.org/r/2579/#comment8054>

          Is there a reason why this has to be included in code as a built in constraint implementation (the only one), instead of just included in documentation as an example?

          As an example it makes sense to me. As a the only built-in, it doesn't, since the way it requires integers to be stored is at odds with all current usage, as I've previously point out.

          • Gary

          On 2011-11-29 20:19:41, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-29 20:19:41)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 3c12169

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3610 ----------------------------------------------------------- Main concerns as same as previously – the HTableDescriptor changes and IntegerConstraint: 1) HTD changes introduce a dependency on Guava. I thought we had eliminated all client-side dependencies on Guava? 2) I think serializing constraint configurations as XML would be useful, as they would then be human readable and require less care to preserve previous serialization formats on upgrade when the property implementations change. 3) I'm still not seeing the use of IntegerConstraint as the one Constraint implementation that we ship as part of core (as opposed to it being an example for documentation). Is there a specific use where you're targeting this? src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2579/#comment8047 > I still don't think this gives us true encapsulation, since the HTD.setValue() allows arbitrary byte[] for values, so there's no way of knowing if (DE)SERIALIZE_KEY was actually used. So is this really better than Bytes.toString()? I also thought that we had eliminated Guava as a client side dependency? This would re-introduce it. That seems the bigger concern unless we've agreed to a client-side Guava dependency somewhere. Why not handle the values as Strings and serialize the Contraint configuration objects as XML? In this case, I think human readable is actually good. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java < https://reviews.apache.org/r/2579/#comment8059 > Omit copyright line src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8058 > Omit copyright line src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8055 > Should constraints apply to Increments and Deletes as well? I don't know if they should or not, and even if they do, it doesn't need to be part of this issue. Just something to consider. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java < https://reviews.apache.org/r/2579/#comment8057 > Omit copyright line src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment8056 > Copyright line not necessary in file headers. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8060 > Having HTD.getValue() convert to String here and then immediately converting back to byte[] is odd. I think either add HTD.getValueAsBytes() returning the raw byte[] or serialize the Configuration as XML as I suggested and just handle it as a String here. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8053 > The manual serialization of properties here seems likely to be brittle and subject to breakage on upgrade if the implementation changes (unless backwards compatibility is explicitly accounted for). This actually seems like a decent case for using the Configuration class' XML serialization support. The output will be much larger, but more resistant to changes in the properties serialized. Just clone the input configuration object, set the "enabled" and "priority" properties under known keys, and serialize to a string using Configuration.writeXML(). src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java < https://reviews.apache.org/r/2579/#comment8054 > Is there a reason why this has to be included in code as a built in constraint implementation (the only one), instead of just included in documentation as an example? As an example it makes sense to me. As a the only built-in, it doesn't, since the way it requires integers to be stored is at odds with all current usage, as I've previously point out. Gary On 2011-11-29 20:19:41, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-29 20:19:41) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 3c12169 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Ted Yu added a comment -

          Thanks Gary for the detailed review.
          For the check() API:

            public void check(Put p) throws ConstraintException;
          

          We can abstract the input parameter as Mutation.

          I think we need to reach agreement on the following major issues:
          1. introduction of dependency of Guava in client library - Jonathan Gray would strongly disagree with this practice.
          2. whether IntegerConstraint should be included in the constraint package
          3. constraint configuration serialization method

          If the current implementation for some of the above isn't critical to this feature (#1 and #2), we can defer to future JIRAs.

          Actually a fourth issue (raised by Suraj under the discussion of 'Question on Coprocessors and Atomicity') is more important: if we don't provide atomicity by holding rowlock during the check, the use cases for Constraints would decrease.
          Issue #4 definitely doesn't have to be covered by this JIRA.

          The fifth issue is how IntegrationTestConstraint.java should be tagged.

          Show
          Ted Yu added a comment - Thanks Gary for the detailed review. For the check() API: public void check(Put p) throws ConstraintException; We can abstract the input parameter as Mutation. I think we need to reach agreement on the following major issues: 1. introduction of dependency of Guava in client library - Jonathan Gray would strongly disagree with this practice. 2. whether IntegerConstraint should be included in the constraint package 3. constraint configuration serialization method If the current implementation for some of the above isn't critical to this feature (#1 and #2), we can defer to future JIRAs. Actually a fourth issue (raised by Suraj under the discussion of 'Question on Coprocessors and Atomicity') is more important: if we don't provide atomicity by holding rowlock during the check, the use cases for Constraints would decrease. Issue #4 definitely doesn't have to be covered by this JIRA. The fifth issue is how IntegrationTestConstraint.java should be tagged.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3653
          -----------------------------------------------------------

          Answering most of the big issues on HBASE-4605 since there is a clean summary there by Ted.

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          <https://reviews.apache.org/r/2579/#comment8150>

          Done

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8151>

          Done

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8153>

          this could definitely be an extension. +1 on future considerations.

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
          <https://reviews.apache.org/r/2579/#comment8152>

          Done

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          <https://reviews.apache.org/r/2579/#comment8154>

          done

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8147>

          I think we pulled getValueAsBytes() from HTD since this was the only use case. Agreed that it is a bit awkward, but we figured that since this is really the only use case for it, it didn't make sense to keep that function around.

          Adding this back in makes it easier to remove guava, since that ser/deser can be kept entirely under the covers.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8148>

          +1 on using the configuration.

          Originally, this was done to ensure that the configuration could be completely open to the user, but constraining (no pun intended) the conf by a couple values is not a big deal.

          However, again we are at the question of keeping it human readable or not. See top comment.

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java
          <https://reviews.apache.org/r/2579/#comment8149>

          Moved to documentation.

          • Jesse

          On 2011-11-29 20:19:41, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-11-29 20:19:41)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 3c12169

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3653 ----------------------------------------------------------- Answering most of the big issues on HBASE-4605 since there is a clean summary there by Ted. src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java < https://reviews.apache.org/r/2579/#comment8150 > Done src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8151 > Done src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8153 > this could definitely be an extension. +1 on future considerations. src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java < https://reviews.apache.org/r/2579/#comment8152 > Done src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java < https://reviews.apache.org/r/2579/#comment8154 > done src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8147 > I think we pulled getValueAsBytes() from HTD since this was the only use case. Agreed that it is a bit awkward, but we figured that since this is really the only use case for it, it didn't make sense to keep that function around. Adding this back in makes it easier to remove guava, since that ser/deser can be kept entirely under the covers. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8148 > +1 on using the configuration. Originally, this was done to ensure that the configuration could be completely open to the user, but constraining (no pun intended) the conf by a couple values is not a big deal. However, again we are at the question of keeping it human readable or not. See top comment. src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java < https://reviews.apache.org/r/2579/#comment8149 > Moved to documentation. Jesse On 2011-11-29 20:19:41, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-11-29 20:19:41) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 3c12169 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/IntegerConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/IntegrationTestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestIntegerConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          @Gary:
          Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default.

          getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution.

          Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style.
          How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value.

          check(Put) - this could definitely be an extension. +1 on future considerations.

          IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs

          @Ted:

          IntegrationTestConstraint - based on the recent discussion on dev@ about testing, this should be renamed to just TestConstraint and labeled as @MediumTest. The rest should be moved to @SmallTest

          Atomicity - since per-row modifications are done essentially serially on the region, is this really that big of an issue? If not, then another jira isn't a necessary, but maybe something to consider going forward. Taking the row lock is going to be a big slowdown - it just depends on what the common use case is going to be.

          (this is reproducing a some of what was commented on RB, but is nice to have succinctly).

          Show
          Jesse Yates added a comment - @Gary: Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default. getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution. Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style. How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value. check(Put) - this could definitely be an extension. +1 on future considerations. IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs @Ted: IntegrationTestConstraint - based on the recent discussion on dev@ about testing, this should be renamed to just TestConstraint and labeled as @MediumTest. The rest should be moved to @SmallTest Atomicity - since per-row modifications are done essentially serially on the region, is this really that big of an issue? If not, then another jira isn't a necessary, but maybe something to consider going forward. Taking the row lock is going to be a big slowdown - it just depends on what the common use case is going to be. (this is reproducing a some of what was commented on RB, but is nice to have succinctly).
          Hide
          Ted Yu added a comment -

          Renaming IntegrationTestConstraint as described above makes sense.

          Thanks Jesse.

          Show
          Ted Yu added a comment - Renaming IntegrationTestConstraint as described above makes sense. Thanks Jesse.
          Hide
          Ted Yu added a comment -

          w.r.t. atomicity, it is outside the initial scope of this JIRA.

          Show
          Ted Yu added a comment - w.r.t. atomicity, it is outside the initial scope of this JIRA.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 18 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -160 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/465//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/465//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/465//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -160 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 74 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/465//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/465//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/465//console This message is automatically generated.
          Jesse Yates made changes -
          Link This issue blocks HBASE-4999 [ HBASE-4999 ]
          Hide
          Gary Helmling added a comment -

          Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default.

          getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution.

          Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style.
          How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value.

          Wouldn't all of these parts be solved by just serializing the Configuration as XML? Then it's treated as a string and you don't need the HTD changes that introduce Guava, or getValueAsBytes(), and you get human readability as a nice side-effect. I don't see any benefit to serializing Configuration as a Writable, given these issues.

          check(Put) - this could definitely be an extension. +1 on future considerations.

          I agree that this extending constraints to cover Delete and Increment could be handled as additional issues. I just wanted to raise the question. It's worth thinking about here, since this is introducing a new client-exposed API. But there's still time to make additions if desired prior to 0.94.

          IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs

          Thanks, I think it's a valuable example.

          Show
          Gary Helmling added a comment - Guava- I didn't know about the effort to get rid of guava client side, it can be pulled if it is really a big issue. I just read through 3264 - is it the case that only the client side dependencies are used for the MR stuff? For this it works out really nicely for doing the auto-serializing (see changes to the CP stuff). We could say that the set bytes for key is at your own risk, and make strings the default. getValueAsBytes() - this was -1'ed in a previous iteration as constraints was the only use case for it - wrapping it with the explicit functions seemed a decent solution. Configuration - As far as human readable, how often are people going to need to check those values? The HTD already just keeps around bytes, so people don't check those over the wire - constraints would remain in the same style. How would you feel about making that configurable? I'm thinking setting a debug flag in Constraints/configuration for that value. Wouldn't all of these parts be solved by just serializing the Configuration as XML? Then it's treated as a string and you don't need the HTD changes that introduce Guava, or getValueAsBytes(), and you get human readability as a nice side-effect. I don't see any benefit to serializing Configuration as a Writable, given these issues. check(Put) - this could definitely be an extension. +1 on future considerations. I agree that this extending constraints to cover Delete and Increment could be handled as additional issues. I just wanted to raise the question. It's worth thinking about here, since this is introducing a new client-exposed API. But there's still time to make additions if desired prior to 0.94. IntegerConstraint - In the end, I can let this go. You're right that having in the docs should be enough. Also, combined with the fact that we don't have any others by default (and that removing is far harder than adding) I'll drop it into just the docs Thanks, I think it's a valuable example.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/
          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03.559611)

          Review request for hbase.

          Changes
          -------

          Making changes as per last comments on last diff:
          1) Removing copywrite lines
          2) Adding test categorization
          3) Removing guava from HTD
          4) Serializing conf to xml
          5) Moving integerConstraint to package-info

          Summary
          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.
          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs (updated)


          src/docbkx/book.xml 9617950
          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a
          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION
          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION
          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing
          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03.559611) Review request for hbase. Changes ------- Making changes as per last comments on last diff: 1) Removing copywrite lines 2) Adding test categorization 3) Removing guava from HTD 4) Serializing conf to xml 5) Moving integerConstraint to package-info Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs (updated) src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          For Hadoop-QA

          Show
          Jesse Yates added a comment - For Hadoop-QA
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v5.patch [ 12507251 ]
          Hide
          Ted Yu added a comment -

          The above message from QA meant that the patch must be applied with -p1.

          I tried to produce a patch which can be applied with -p0 but I got:

          patching file src/docbkx/book.xml
          Reversed (or previously applied) patch detected!  Assume -R? [n] 
          Apply anyway? [n] 
          Skipping patch.
          1 out of 1 hunk ignored -- saving rejects to file src/docbkx/book.xml.rej
          

          @Jesse:
          Do you mind refreshing your patch using --no-prefix ?

          Thanks

          Show
          Ted Yu added a comment - The above message from QA meant that the patch must be applied with -p1. I tried to produce a patch which can be applied with -p0 but I got: patching file src/docbkx/book.xml Reversed (or previously applied) patch detected! Assume -R? [n] Apply anyway? [n] Skipping patch. 1 out of 1 hunk ignored -- saving rejects to file src/docbkx/book.xml.rej @Jesse: Do you mind refreshing your patch using --no-prefix ? Thanks
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3897
          -----------------------------------------------------------

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8770>

          Multiple Constraints are supported. Please update javadoc.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8772>

          Priority is updated at the end.
          I think we should document concurrency assumption in javadoc.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8768>

          Can we call ByteArrayOutputStream.toString() directly ?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment8769>

          The dash is not needed.

          • Ted

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3897 ----------------------------------------------------------- src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8770 > Multiple Constraints are supported. Please update javadoc. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8772 > Priority is updated at the end. I think we should document concurrency assumption in javadoc. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8768 > Can we call ByteArrayOutputStream.toString() directly ? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment8769 > The dash is not needed. Ted On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          Ok, this time --no-prefix (duh).

          Show
          Jesse Yates added a comment - Ok, this time --no-prefix (duh).
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v5.patch [ 12507271 ]
          Ted Yu made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Ted Yu made changes -
          Attachment java_HBASE-4605_v5.patch [ 12507251 ]
          Ted Yu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12507271/java_HBASE-4605_v5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -150 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestAdmin

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/501//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/501//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/501//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507271/java_HBASE-4605_v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -150 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/501//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/501//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/501//console This message is automatically generated.
          Hide
          Jesse Yates added a comment -

          Hmm, didn't touch any of the admin code, but it looks like its another instance of the 'too many open files' issue coming up again.

          No new warnings on any of the code in this patch (as far as I could tell).

          Show
          Jesse Yates added a comment - Hmm, didn't touch any of the admin code, but it looks like its another instance of the 'too many open files' issue coming up again. No new warnings on any of the code in this patch (as far as I could tell).
          Hide
          Ted Yu added a comment -

          TestHTableDescriptor misses category.

          Other than that I vote +1 on latest patch.

          Show
          Ted Yu added a comment - TestHTableDescriptor misses category. Other than that I vote +1 on latest patch.
          Hide
          Ted Yu added a comment -

          Patch that adds category to TestHTableDescriptor

          I plan to integrate v6 if there is no objection from Gary.

          Show
          Ted Yu added a comment - Patch that adds category to TestHTableDescriptor I plan to integrate v6 if there is no objection from Gary.
          Ted Yu made changes -
          Attachment 4605-v6.txt [ 12507384 ]
          Hide
          Jesse Yates added a comment -

          I'd like to get the comments updated as per the latest comments on RB. Otherwise, +1

          Show
          Jesse Yates added a comment - I'd like to get the comments updated as per the latest comments on RB. Otherwise, +1
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12507384/4605-v6.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 24 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -150 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestAdmin
          org.apache.hadoop.hbase.client.TestInstantSchemaChange

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/506//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/506//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/506//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507384/4605-v6.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 24 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -150 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestInstantSchemaChange Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/506//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/506//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/506//console This message is automatically generated.
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-14 00:40:59, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 335

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line335>

          >

          > Can we call ByteArrayOutputStream.toString() directly ?

          No, because there is no assurance which encoding will be used. This way, we can be sure that it is UTF-8 encoded and conforms to the rest of the project. Also, the overhead here is just one more, short-lived pointer (not a big deal).

          On 2011-12-14 00:40:59, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 196

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line196>

          >

          > Priority is updated at the end.

          > I think we should document concurrency assumption in javadoc.

          Added a section to package-info talking about concurrency.

          On 2011-12-14 00:40:59, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 179

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line179>

          >

          > Multiple Constraints are supported. Please update javadoc.

          done.

          On 2011-12-14 00:40:59, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 355

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line355>

          >

          > The dash is not needed.

          done.

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3897
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-14 00:40:59, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 335 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line335 > > > Can we call ByteArrayOutputStream.toString() directly ? No, because there is no assurance which encoding will be used. This way, we can be sure that it is UTF-8 encoded and conforms to the rest of the project. Also, the overhead here is just one more, short-lived pointer (not a big deal). On 2011-12-14 00:40:59, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 196 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line196 > > > Priority is updated at the end. > I think we should document concurrency assumption in javadoc. Added a section to package-info talking about concurrency. On 2011-12-14 00:40:59, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 179 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line179 > > > Multiple Constraints are supported. Please update javadoc. done. On 2011-12-14 00:40:59, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 355 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line355 > > > The dash is not needed. done. Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3897 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          Jesse Yates added a comment -

          Updated patch with fixed javadocs as per Ted's comments.

          Show
          Jesse Yates added a comment - Updated patch with fixed javadocs as per Ted's comments.
          Jesse Yates made changes -
          Attachment java_HBASE-4605_v7.patch [ 12507427 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12507427/java_HBASE-4605_v7.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 16 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated -152 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestAdmin
          org.apache.hadoop.hbase.client.TestInstantSchemaChange

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/508//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/508//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/508//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507427/java_HBASE-4605_v7.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -152 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 75 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestInstantSchemaChange Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/508//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/508//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/508//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          Test failures were due to 'Too many open files'

          Will commit tomorrow.

          Show
          Ted Yu added a comment - Test failures were due to 'Too many open files' Will commit tomorrow.
          Ted Yu made changes -
          Comment [ -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12504935/4605.v7
            against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 18 new or modified tests.

              -1 javadoc. The javadoc tool appears to have generated -162 warning messages.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              -1 findbugs. The patch appears to introduce 68 new Findbugs (version 1.3.9) warnings.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

               -1 core tests. The patch failed these unit tests:
                                 org.apache.hadoop.hbase.replication.TestReplication
                            org.apache.hadoop.hbase.client.TestAdmin
                            org.apache.hadoop.hbase.client.TestInstantSchemaChange

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/352//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/352//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/352//console

          This message is automatically generated. ]
          Ted Yu made changes -
          Comment [ -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12505275/java_HBASE-4605_v1.patch
            against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 18 new or modified tests.

              -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/384//console

          This message is automatically generated. ]
          Ted Yu made changes -
          Comment [ -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12505277/java_HBASE-4605_v2.patch
            against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 18 new or modified tests.

              -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/385//console

          This message is automatically generated. ]
          Ted Yu made changes -
          Comment [ -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12507251/java_HBASE-4605_v5.patch
            against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 16 new or modified tests.

              -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/500//console

          This message is automatically generated. ]
          Hide
          Ted Yu added a comment -

          Integrated to TRUNK.

          Thanks for the patch Jesse.

          Thanks for the review Gary.

          Show
          Ted Yu added a comment - Integrated to TRUNK. Thanks for the patch Jesse. Thanks for the review Gary.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2548 (See https://builds.apache.org/job/HBase-TRUNK/2548/)
          HBASE-4605 Constraints, remove .orig files accidentally added in previous commit
          HBASE-4605 Constraints (Jesse Yates)

          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig

          tedyu :
          Files :

          • /hbase/trunk/src/docbkx/book.xml
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2548 (See https://builds.apache.org/job/HBase-TRUNK/2548/ ) HBASE-4605 Constraints, remove .orig files accidentally added in previous commit HBASE-4605 Constraints (Jesse Yates) tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig tedyu : Files : /hbase/trunk/src/docbkx/book.xml /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #33 (See https://builds.apache.org/job/HBase-TRUNK-security/33/)
          HBASE-4605 Constraints, remove .orig files accidentally added in previous commit
          HBASE-4605 Constraints (Jesse Yates)

          tedyu :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig

          tedyu :
          Files :

          • /hbase/trunk/src/docbkx/book.xml
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #33 (See https://builds.apache.org/job/HBase-TRUNK-security/33/ ) HBASE-4605 Constraints, remove .orig files accidentally added in previous commit HBASE-4605 Constraints (Jesse Yates) tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig tedyu : Files : /hbase/trunk/src/docbkx/book.xml /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java.orig /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/constraint/package-info.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java.orig /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java.orig
          Ted Yu made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Hide
          stack added a comment -

          Did this patch add guava as a client-side dependency in the end?

          Doc needs boosting for this new feature; as is, a bunch of detail on howto is missing.

          IMO, this patch could have baked out one RB a little longer. Its a new feature. I didn't realize it going in (was slow to get around to review). I'd like to have gone over it before it went in. I could review now I suppose but feels a bit late...

          Show
          stack added a comment - Did this patch add guava as a client-side dependency in the end? Doc needs boosting for this new feature; as is, a bunch of detail on howto is missing. IMO, this patch could have baked out one RB a little longer. Its a new feature. I didn't realize it going in (was slow to get around to review). I'd like to have gone over it before it went in. I could review now I suppose but feels a bit late...
          Hide
          Ted Yu added a comment -

          Guava dependency has been removed.

          Feel free to provide your comments. I and Jesse would accommodate them.

          Show
          Ted Yu added a comment - Guava dependency has been removed. Feel free to provide your comments. I and Jesse would accommodate them.
          Hide
          Jesse Yates added a comment -

          @Stack: yeah, we can definitely made mods later. As far as docs go, there is extensive information in the package-info. If you want we can also port/copy it to the book or have a comment in the book about looking at the package info.

          Show
          Jesse Yates added a comment - @Stack: yeah, we can definitely made mods later. As far as docs go, there is extensive information in the package-info. If you want we can also port/copy it to the book or have a comment in the book about looking at the package info.
          Hide
          Jesse Yates added a comment -

          @stack: and we kept guava out still and basically went back to the original HTD + removeCoprocessor()

          Show
          Jesse Yates added a comment - @stack: and we kept guava out still and basically went back to the original HTD + removeCoprocessor()
          Hide
          stack added a comment -

          Ok. Thanks lads. Good there is not extra dependency client-side (its a pain MR'ing)

          I'll do a bit of review.....

          Show
          stack added a comment - Ok. Thanks lads. Good there is not extra dependency client-side (its a pain MR'ing) I'll do a bit of review.....
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3980
          -----------------------------------------------------------

          Very nice. Some comments below.

          src/docbkx/book.xml
          <https://reviews.apache.org/r/2579/#comment8986>

          This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here?

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2579/#comment8987>

          We're adding white space here

          Should there be an answering String add method?

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java
          <https://reviews.apache.org/r/2579/#comment8988>

          This just removes cp from HTD list. If loaded on a regionserver, it continues to be so?

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          <https://reviews.apache.org/r/2579/#comment8989>

          Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package?

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java
          <https://reviews.apache.org/r/2579/#comment8990>

          Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8994>

          Nit: Missing 'of', 'not' should be 'no'

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8995>

          'not' should be 'no'

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8996>

          Missing 'should'?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8997>

          Nice class doc

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment8998>

          I can't have a constraint on a Delete? Should this be a Mutation?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9003>

          key for sure is not going to be null?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9002>

          Style: Braces if on separate line. No need of braces if 'if' test and if statement are on same line. In fact this could have been written:

          return value == null? null: new Pair<String, String>(key, value);

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9006>

          This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9007>

          Ain't these really fat? Ain't this going to bloat HTD?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9008>

          This method is about Strings, not byte arrays?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9010>

          This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9012>

          Incomplete sentence

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9014>

          This is expensive call. Its done infrequently?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9015>

          Can I ask whether a constraint enabled or disabled?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9016>

          Braces.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9017>

          For sure not null?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9026>

          Where do I get one of these? Which classloader to use?

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9027>

          Should this be a priority comparator rather than a constraintcomparator?

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment9028>

          Should there be protection against constraints being unloaded? For an administrator only?

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment9029>

          This text could do w/ a read-through/edit (see things like the curly-brace here)

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment9032>

          Do I have to alter the table after doing this?

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment9033>

          Need a rolling restart

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java
          <https://reviews.apache.org/r/2579/#comment9034>

          Is above missing the reenable of table after HTD editing?

          This doc is excellent. Needs a pointer from book to here.

          • Michael

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3980 ----------------------------------------------------------- Very nice. Some comments below. src/docbkx/book.xml < https://reviews.apache.org/r/2579/#comment8986 > This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here? src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2579/#comment8987 > We're adding white space here Should there be an answering String add method? src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java < https://reviews.apache.org/r/2579/#comment8988 > This just removes cp from HTD list. If loaded on a regionserver, it continues to be so? src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java < https://reviews.apache.org/r/2579/#comment8989 > Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package? src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java < https://reviews.apache.org/r/2579/#comment8990 > Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html? src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8994 > Nit: Missing 'of', 'not' should be 'no' src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8995 > 'not' should be 'no' src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8996 > Missing 'should'? src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8997 > Nice class doc src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment8998 > I can't have a constraint on a Delete? Should this be a Mutation? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9003 > key for sure is not going to be null? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9002 > Style: Braces if on separate line. No need of braces if 'if' test and if statement are on same line. In fact this could have been written: return value == null? null: new Pair<String, String>(key, value); src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9006 > This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9007 > Ain't these really fat? Ain't this going to bloat HTD? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9008 > This method is about Strings, not byte arrays? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9010 > This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9012 > Incomplete sentence src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9014 > This is expensive call. Its done infrequently? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9015 > Can I ask whether a constraint enabled or disabled? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9016 > Braces. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9017 > For sure not null? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9026 > Where do I get one of these? Which classloader to use? src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9027 > Should this be a priority comparator rather than a constraintcomparator? src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment9028 > Should there be protection against constraints being unloaded? For an administrator only? src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment9029 > This text could do w/ a read-through/edit (see things like the curly-brace here) src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment9032 > Do I have to alter the table after doing this? src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment9033 > Need a rolling restart src/main/java/org/apache/hadoop/hbase/constraint/package-info.java < https://reviews.apache.org/r/2579/#comment9034 > Is above missing the reenable of table after HTD editing? This doc is excellent. Needs a pointer from book to here. Michael On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3990
          -----------------------------------------------------------

          Thanks for the valuable comments, Stack.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java
          <https://reviews.apache.org/r/2579/#comment9036>

          I was thinking about this too.
          Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9035>

          I wish we had brought up this argument to Gary earlier.

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java
          <https://reviews.apache.org/r/2579/#comment9037>

          Currently this method is called by testUpdateConstraint()

          • Ted

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3990 ----------------------------------------------------------- Thanks for the valuable comments, Stack. src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java < https://reviews.apache.org/r/2579/#comment9036 > I was thinking about this too. Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9035 > I wish we had brought up this argument to Gary earlier. src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java < https://reviews.apache.org/r/2579/#comment9037 > Currently this method is called by testUpdateConstraint() Ted On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-20 00:20:28, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>

          >

          > I wish we had brought up this argument to Gary earlier.

          I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:

          1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies
          2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.

          • Gary

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3990
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-20 00:20:28, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278 > > > I wish we had brought up this argument to Gary earlier. I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are: 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants. Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3990 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Ted Yu made changes -
          Link This issue is related to HBASE-5070 [ HBASE-5070 ]
          Hide
          Ted Yu added a comment -

          I created HBASE-5070 where further code changes should be made.

          Show
          Ted Yu added a comment - I created HBASE-5070 where further code changes should be made.
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 413

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line413>

          >

          > This is expensive call. Its done infrequently?

          Essentially, only when we update the configuration for a constraint - adding, changing enabled or not, setting priority. The alternative was to keep around the those values just as bytes in the value, but Gary's argument for keeping them in the configuration for ease and compatibility going forward was the basis for this change. These confs are expected to be very small, so this shouldn't really be that bad to write.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 43

          > <https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line43>

          >

          > Should there be protection against constraints being unloaded? For an administrator only?

          Are there ways we can enforce that just admins can do certain things? Also, I feel like if people are creating their own tables, they should be free to modify their own tables. Clearly this gets into the security stuff of which tables you should able to modify, which could be a bigger proposition. Thoughts?

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 479

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line479>

          >

          > Can I ask whether a constraint enabled or disabled?

          Yup - see enabled(HTD, Constraint).

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>

          >

          > Ain't these really fat? Ain't this going to bloat HTD?

          Yeah, it will bloat the HTD. Let's move this discussion down to comments on Ted's review or onto 5070.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 166

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line166>

          >

          > key for sure is not going to be null?

          it'll propagate a null pointer from the from the class->key conversion before the return value here is null. What's the standard on handling the null pointer issues client side? Ok to just let that propogate up? Defensive checks? Wrapping it in some other exception? ??

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 1

          > <https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line1>

          >

          > Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package?

          Having it in its own package was part of what I would consider making constraints a "top-level" feature. It really depends on how we want to handle doing constraints and CP related features going forward. For instance, the security stuff is not only its own package, its going into its own module. I would argue that constraints are less key than security, but important enough to warrant its own subsection. Otherwise, we can get into a situation where we either have a ton of classes in the coprocessors package (bad) or a ton of packages - for each new feature (potentially way worse). I think we need to treat them on a case-by-case basis going forward.

          There are significant docs on constraints, making them not a great thing to just drop into the coprocessor folder. Maybe that can be the metric - number of lines of necessary documentation (as opposed to just all docs, to avoid people creating filler just to have its own package).

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/docbkx/book.xml, line 874

          > <https://reviews.apache.org/r/2579/diff/8/?file=64467#file64467line874>

          >

          > This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here?

          Should the link just be to the javadocs (which we could forget to update when this release gets cut) or should it just be a comment that usage is further explained in the javadocs/package-info?

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 480

          > <https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line480>

          >

          > We're adding white space here

          >

          > Should there be an answering String add method?

          We don't have an addValue for bytes, so I figured it was unnecessary.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 299

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line299>

          >

          > This method is about Strings, not byte arrays?

          yup, this is from an earlier impl

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 588

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line588>

          >

          > Should this be a priority comparator rather than a constraintcomparator?

          Internally, it is comparing the priorities, but those are local to the constraints. What we really care about in the end is the order the constraints are applied, hence, constraintComparator. But priority comparator could work too.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 65

          > <https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line65>

          >

          > This text could do w/ a read-through/edit (see things like the curly-brace here)

          hmmm, that line didn't make it into trunk, but giving it another once-over.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 26

          > <https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line26>

          >

          > Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html?

          Even more so, probably Configured? Kinda makes this class a little obsolete... I'm partial to keeping it in there, just to tie the two hierarchies together - by implemnting BaseConstraint, all they need to take care of is overriding the methods specified.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 173

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line173>

          >

          > This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding?

          Its in the package-info, but ok, here works too.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 317

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line317>

          >

          > This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it?

          Yeah, but there isn't a really good way of getting them across otherwise. Follow-up should be in 5070 and to ted's comment below.

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1072

          > <https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line1072>

          >

          > This just removes cp from HTD list. If loaded on a regionserver, it continues to be so?

          I believe so - you should have to disable and then renable the table for it to take effect (though it may be that you actually need to roll the RS).

          On 2011-12-19 23:56:21, Michael Stack wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 518

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line518>

          >

          > For sure not null?

          yes...private method, so we should be able to expect valid elements - people shouldn't be able to mess with the values here/put bad data.

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3980
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 413 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line413 > > > This is expensive call. Its done infrequently? Essentially, only when we update the configuration for a constraint - adding, changing enabled or not, setting priority. The alternative was to keep around the those values just as bytes in the value, but Gary's argument for keeping them in the configuration for ease and compatibility going forward was the basis for this change. These confs are expected to be very small, so this shouldn't really be that bad to write. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 43 > < https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line43 > > > Should there be protection against constraints being unloaded? For an administrator only? Are there ways we can enforce that just admins can do certain things? Also, I feel like if people are creating their own tables, they should be free to modify their own tables. Clearly this gets into the security stuff of which tables you should able to modify, which could be a bigger proposition. Thoughts? On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 479 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line479 > > > Can I ask whether a constraint enabled or disabled? Yup - see enabled(HTD, Constraint). On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278 > > > Ain't these really fat? Ain't this going to bloat HTD? Yeah, it will bloat the HTD. Let's move this discussion down to comments on Ted's review or onto 5070. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 166 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line166 > > > key for sure is not going to be null? it'll propagate a null pointer from the from the class->key conversion before the return value here is null. What's the standard on handling the null pointer issues client side? Ok to just let that propogate up? Defensive checks? Wrapping it in some other exception? ?? On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 1 > < https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line1 > > > Constraints are in a constraint package? This sets up a precedent where each cp gets its own package? Can't we have constraint in coprocessor package? Having it in its own package was part of what I would consider making constraints a "top-level" feature. It really depends on how we want to handle doing constraints and CP related features going forward. For instance, the security stuff is not only its own package, its going into its own module. I would argue that constraints are less key than security, but important enough to warrant its own subsection. Otherwise, we can get into a situation where we either have a ton of classes in the coprocessors package (bad) or a ton of packages - for each new feature (potentially way worse). I think we need to treat them on a case-by-case basis going forward. There are significant docs on constraints, making them not a great thing to just drop into the coprocessor folder. Maybe that can be the metric - number of lines of necessary documentation (as opposed to just all docs, to avoid people creating filler just to have its own package). On 2011-12-19 23:56:21, Michael Stack wrote: > src/docbkx/book.xml, line 874 > < https://reviews.apache.org/r/2579/diff/8/?file=64467#file64467line874 > > > This needs filling out some; how do I add a constraint. Perhaps this is in package info? If so, add link to it from here? Should the link just be to the javadocs (which we could forget to update when this release gets cut) or should it just be a comment that usage is further explained in the javadocs/package-info? On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 480 > < https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line480 > > > We're adding white space here > > Should there be an answering String add method? We don't have an addValue for bytes, so I figured it was unnecessary. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 299 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line299 > > > This method is about Strings, not byte arrays? yup, this is from an earlier impl On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 588 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line588 > > > Should this be a priority comparator rather than a constraintcomparator? Internally, it is comparing the priorities, but those are local to the constraints. What we really care about in the end is the order the constraints are applied, hence, constraintComparator. But priority comparator could work too. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/package-info.java, line 65 > < https://reviews.apache.org/r/2579/diff/8/?file=64474#file64474line65 > > > This text could do w/ a read-through/edit (see things like the curly-brace here) hmmm, that line didn't make it into trunk, but giving it another once-over. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java, line 26 > < https://reviews.apache.org/r/2579/diff/8/?file=64469#file64469line26 > > > Should have used http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/conf/Configurable.html? Even more so, probably Configured? Kinda makes this class a little obsolete... I'm partial to keeping it in there, just to tie the two hierarchies together - by implemnting BaseConstraint, all they need to take care of is overriding the methods specified. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 173 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line173 > > > This method is about adding constraints but the body of the method seems to be about upping priority. Should explain what priority upping has to do with constraint adding? Its in the package-info, but ok, here works too. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 317 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line317 > > > This is going to serialize a fat Configuration into an HTD? If I list an HTD in shell, its going to have this Configuration pollution in it? Yeah, but there isn't a really good way of getting them across otherwise. Follow-up should be in 5070 and to ted's comment below. On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java, line 1072 > < https://reviews.apache.org/r/2579/diff/8/?file=64468#file64468line1072 > > > This just removes cp from HTD list. If loaded on a regionserver, it continues to be so? I believe so - you should have to disable and then renable the table for it to take effect (though it may be that you actually need to roll the RS). On 2011-12-19 23:56:21, Michael Stack wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 518 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line518 > > > For sure not null? yes...private method, so we should be able to expect valid elements - people shouldn't be able to mess with the values here/put bad data. Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3980 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-20 00:20:28, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 73

          > <https://reviews.apache.org/r/2579/diff/8/?file=64470#file64470line73>

          >

          > I was thinking about this too.

          > Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint.

          If you look at HBASE-4999 you could, theoretically, break a constraint via a delete, since it considers the state of the underlying table. However, that is a later consideration. We already talked about moving Constraints to modifying Mutations in a later ticket.

          On 2011-12-20 00:20:28, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>

          >

          > I wish we had brought up this argument to Gary earlier.

          Gary Helmling wrote:

          I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:

          1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies

          2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.

          What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?

          On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3990
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-20 00:20:28, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java, line 73 > < https://reviews.apache.org/r/2579/diff/8/?file=64470#file64470line73 > > > I was thinking about this too. > Since the current Constraint is for one table only, I couldn't see how deletion of certain row would break constraint. If you look at HBASE-4999 you could, theoretically, break a constraint via a delete, since it considers the state of the underlying table. However, that is a later consideration. We already talked about moving Constraints to modifying Mutations in a later ticket. On 2011-12-20 00:20:28, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278 > > > I wish we had brought up this argument to Gary earlier. Gary Helmling wrote: I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are: 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants. What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070 . Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data? On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side". Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3990 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-20 00:20:28, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>

          >

          > I wish we had brought up this argument to Gary earlier.

          Gary Helmling wrote:

          I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:

          1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies

          2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.

          Jesse Yates wrote:

          What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?

          On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".

          Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it?

          • Michael

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3990
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-20 00:20:28, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278 > > > I wish we had brought up this argument to Gary earlier. Gary Helmling wrote: I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are: 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants. Jesse Yates wrote: What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070 . Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data? On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side". Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it? Michael ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3990 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-20 00:20:28, Ted Yu wrote:

          > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278

          > <https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278>

          >

          > I wish we had brought up this argument to Gary earlier.

          Gary Helmling wrote:

          I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are:

          1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies

          2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants.

          Jesse Yates wrote:

          What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070. Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data?

          On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side".

          Michael Stack wrote:

          Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it?

          Not yet - I was putting all the shell work off until 4879. Figured if it looked gross/crazy, we could handle it later. It also depends on what people are actually doing with the configuration stuff - if its not much, then this point is really moot - we can't really tell until people try it out and complain (or not smile).

          • Jesse

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2579/#review3990
          -----------------------------------------------------------

          On 2011-12-13 21:38:03, Jesse Yates wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2579/

          -----------------------------------------------------------

          (Updated 2011-12-13 21:38:03)

          Review request for hbase.

          Summary

          -------

          Most of the implementation for adding constraints as a coprocessor.

          Looking for general comments on style/structure, though nitpicks are ok too.

          Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good).

          This addresses bug HBASE-4605.

          https://issues.apache.org/jira/browse/HBASE-4605

          Diffs

          -----

          src/docbkx/book.xml 9617950

          src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a

          src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION

          src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION

          src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2579/diff

          Testing

          -------

          Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass.

          Thanks,

          Jesse

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-20 00:20:28, Ted Yu wrote: > src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java, line 278 > < https://reviews.apache.org/r/2579/diff/8/?file=64473#file64473line278 > > > I wish we had brought up this argument to Gary earlier. Gary Helmling wrote: I don't see how stuffing binary data into the HTD values (which is what I argued against) is any less polluting HTD? I think the options here are: 1) store in HTD, in which case I think the current string representation is better than what we had before with the Guava dependencies 2) store configuration for Constraints to the side, in which case ConstraintProcessor can handle mapping configuration to the Contraint instances however it wants. Jesse Yates wrote: What do you mean by "to the side"? Ted mentions a centralized repo in HBASE-5070 . Are you thinking of essentially a system-level table for just this stuff - kind of an aux-meta table that can be used to store just arbitrary data? On one hand cool, on the other, but big are the confs really going to be in the HTD? Probably not more than a couple lines, and you may or may not want to see it, but at the very least you will always get it, as opposed to worrying about another moving part, that may or may not be there "on the side". Michael Stack wrote: Adding conf as String to HTD is for sure convenient. Just wondering what it'll look like when we describe a table. What happens when we want to alter a table? Maybe this a non-issue/just aesthetics. Have you tried it? Not yet - I was putting all the shell work off until 4879. Figured if it looked gross/crazy, we could handle it later. It also depends on what people are actually doing with the configuration stuff - if its not much, then this point is really moot - we can't really tell until people try it out and complain (or not smile). Jesse ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/#review3990 ----------------------------------------------------------- On 2011-12-13 21:38:03, Jesse Yates wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2579/ ----------------------------------------------------------- (Updated 2011-12-13 21:38:03) Review request for hbase. Summary ------- Most of the implementation for adding constraints as a coprocessor. Looking for general comments on style/structure, though nitpicks are ok too. Currently missing implementation for disableConstraints() since that will require adding removeCoprocessor() to HTD (also comments on if this is worth it would be good). This addresses bug HBASE-4605 . https://issues.apache.org/jira/browse/HBASE-4605 Diffs ----- src/docbkx/book.xml 9617950 src/main/java/org/apache/hadoop/hbase/HTableDescriptor.java 84a0d1a src/main/java/org/apache/hadoop/hbase/constraint/BaseConstraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraint.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintException.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/ConstraintProcessor.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/Constraints.java PRE-CREATION src/main/java/org/apache/hadoop/hbase/constraint/package-info.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/TestHTableDescriptor.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/AllPassConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/CheckConfigurationConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/RuntimeFailConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/TestConstraints.java PRE-CREATION src/test/java/org/apache/hadoop/hbase/constraint/WorksConstraint.java PRE-CREATION Diff: https://reviews.apache.org/r/2579/diff Testing ------- Adding IntegrationTestConstraint and unit tests for Constraints and IntegerConstraint. All of those pass. Thanks, Jesse

            People

            • Assignee:
              Jesse Yates
              Reporter:
              Jesse Yates
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development