Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7725

Add ability to create custom compaction request

    Details

    • Release Note:
      Hide
      Enable coprocessors to submit custom compaction requests.

      This compaction request is used (if not null) through the entire compaction process. If no custom compaction request is submitted (its null - the normal case) then we we build a new compaction request via the usual mechanisms in the Store.

      Adding this mechanism is particularly useful when 'special' compactions are desired and you want to do things like: only change certain files for a particularly requested compaction or follow a single compaction through the queue.
      Show
      Enable coprocessors to submit custom compaction requests. This compaction request is used (if not null) through the entire compaction process. If no custom compaction request is submitted (its null - the normal case) then we we build a new compaction request via the usual mechanisms in the Store. Adding this mechanism is particularly useful when 'special' compactions are desired and you want to do things like: only change certain files for a particularly requested compaction or follow a single compaction through the queue.

      Description

      You can request that a compaction be started, but you can't be sure when that compaction request completes. This is a simple update to the CompactionRequest interface and the compact-split thread on the RS that doesn't actually impact the RS exposed interface.

      This is particularly useful for CPs so they can control starting/running a compaction.

      Update - Better description (3/20/13):
      Enable coprocessors to submit custom compaction requests.

      This compaction request is used (if not null) through the entire compaction process. If no custom compaction request is submitted (its null - the normal case) then we we build a new compaction request via the usual mechanisms in the Store.

      Adding this mechanism is particularly useful when 'special' compactions are desired and you want to do things like: only change certain files for a particularly requested compaction or follow a single compaction through the queue.

      1. hbase-7725-v6.patch
        57 kB
        Jesse Yates
      2. hbase-7725-v5.patch
        57 kB
        Jesse Yates
      3. hbase-7725-v4.patch
        54 kB
        Jesse Yates
      4. hbase-7725-v3.patch
        90 kB
        Jesse Yates
      5. hbase-7725-v1.patch
        42 kB
        Jesse Yates
      6. hbase-7725-v0.patch
        42 kB
        Jesse Yates
      7. hbase-7725-addenum-v1.patch
        5 kB
        Jesse Yates
      8. hbase-7725-addenum.patch
        5 kB
        Jesse Yates
      9. hbase-7725_with-attributes-0.94-v1.patch
        43 kB
        Jesse Yates
      10. hbase-7725_with-attributes-0.94-v0.patch
        36 kB
        Jesse Yates
      11. hbase-7725_0.94-v4.patch
        55 kB
        Jesse Yates
      12. hbase-7725_0.94-v3.patch
        53 kB
        Jesse Yates
      13. hbase-7725_0.94-v0.patch
        8 kB
        Jesse Yates
      14. example.java
        3 kB
        Jesse Yates

        Issue Links

          Activity

          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching version for 0.94. I'll do trunk if people are onboard

          Show
          jesse_yates Jesse Yates added a comment - Attaching version for 0.94. I'll do trunk if people are onboard
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Can you show us how the latch would be used ?

          Thanks

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Can you show us how the latch would be used ? Thanks
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching simple example file of using the finished latch on a compactionRequest. Essentially, I want to trigger a compaction and make sure I know when it finishes because I want to setup a special compaction policy, but I don't know when the compaction will run or which one is running or if the one I requested is done yet.

          I had considered making "compactRegionAndBlockUntilDone" a method on HRegionServer that we could just call, but that meant updating RegionServerServices and slightly reworking the calls there that request compactions to just drop the return values (keeping the async nature). So we could move it in there, but since its a small case, I don't know if its worth it yet

          Show
          jesse_yates Jesse Yates added a comment - Attaching simple example file of using the finished latch on a compactionRequest. Essentially, I want to trigger a compaction and make sure I know when it finishes because I want to setup a special compaction policy, but I don't know when the compaction will run or which one is running or if the one I requested is done yet. I had considered making "compactRegionAndBlockUntilDone" a method on HRegionServer that we could just call, but that meant updating RegionServerServices and slightly reworking the calls there that request compactions to just drop the return values (keeping the async nature). So we could move it in there, but since its a small case, I don't know if its worth it yet
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Example makes sense to me.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Example makes sense to me.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Lgtm

          Show
          lhofhansl Lars Hofhansl added a comment - Lgtm
          Hide
          lhofhansl Lars Hofhansl added a comment -

          Discussing this a bit further with Jesse. In addition to this for our use case it would be extremely helpful if we could attach some meta-data (or attributes) to a CompactionRequest and have that available in the coprocessor hooks. At least in preCompactScannerOpen, but probably also in preCompact, postCompact, preCompactSelection, postCompactSelection.

          (The goal is to custom compaction instances in the compaction queue, seems easiest to do this with coprocessors this way.)

          Show
          lhofhansl Lars Hofhansl added a comment - Discussing this a bit further with Jesse. In addition to this for our use case it would be extremely helpful if we could attach some meta-data (or attributes) to a CompactionRequest and have that available in the coprocessor hooks. At least in preCompactScannerOpen, but probably also in preCompact, postCompact, preCompactSelection, postCompactSelection. (The goal is to custom compaction instances in the compaction queue, seems easiest to do this with coprocessors this way.)
          Hide
          lhofhansl Lars Hofhansl added a comment -

          And we'd need that in 0.94 if we wanted to go that route.

          Show
          lhofhansl Lars Hofhansl added a comment - And we'd need that in 0.94 if we wanted to go that route.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Sounds reasonable.

          This would go to 0.94.6, right ?

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Sounds reasonable. This would go to 0.94.6, right ?
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching patch of what it would look like with also passing along attributes.

          Show
          jesse_yates Jesse Yates added a comment - Attaching patch of what it would look like with also passing along attributes.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -
          +  public synchronized List<CompactionRequest> requestCompaction(final HRegion r, final String why,
          +      final Object attribute) throws IOException {
          

          Parameter name should be attributes, right ?

          Would the attributes be String or Map ?

          +  public synchronized List<CompactionRequest> requestCompaction(final HRegion r, final String why,
          +      int p, Object attributes) throws IOException {
          +    List<CompactionRequest> requests = new ArrayList<CompactionRequest>(r.getStores().size());
               for(Store s : r.getStores().values()) {
          -      requestCompaction(r, s, why, p);
          +      CompactionRequest cr = requestCompaction(r, s, why, p, null);
          

          Looks like attributes parameter isn't passed on.

          +   * @return The created {@link CompactionRequest CompactionRequests} or an empty list if no
          +   *         compactions were started
              * @throws IOException
              */
          -  public void requestCompaction(final HRegion r, final String why) throws IOException;
          +  public List<CompactionRequest> requestCompaction(final HRegion r, final String why)
          

          I think the requests would be queued. So please modify the @return.

          It would be nice to put trunk patch on review board.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - + public synchronized List<CompactionRequest> requestCompaction( final HRegion r, final String why, + final Object attribute) throws IOException { Parameter name should be attributes, right ? Would the attributes be String or Map ? + public synchronized List<CompactionRequest> requestCompaction( final HRegion r, final String why, + int p, Object attributes) throws IOException { + List<CompactionRequest> requests = new ArrayList<CompactionRequest>(r.getStores().size()); for (Store s : r.getStores().values()) { - requestCompaction(r, s, why, p); + CompactionRequest cr = requestCompaction(r, s, why, p, null ); Looks like attributes parameter isn't passed on. + * @ return The created {@link CompactionRequest CompactionRequests} or an empty list if no + * compactions were started * @ throws IOException */ - public void requestCompaction( final HRegion r, final String why) throws IOException; + public List<CompactionRequest> requestCompaction( final HRegion r, final String why) I think the requests would be queued. So please modify the @return. It would be nice to put trunk patch on review board.
          Hide
          jesse_yates Jesse Yates added a comment -

          Parameter name should be attributes, right ?

          Yup.

          Would the attributes be String or Map ?

          String will lead to munging and map will often be too heavy weight. I don't want to force it - this is an internal hook, so you know what you are passing - you can cast it when you get it.

          Looks like attributes parameter isn't passed on.

          Good catch.

          I think the requests would be queued. So please modify the @return.

          Definitely.

          It would be nice to put trunk patch on review board.

          Its, unfortunately, going to be more than a little bit different. But yeah, I'll post one up soon

          Show
          jesse_yates Jesse Yates added a comment - Parameter name should be attributes, right ? Yup. Would the attributes be String or Map ? String will lead to munging and map will often be too heavy weight. I don't want to force it - this is an internal hook, so you know what you are passing - you can cast it when you get it. Looks like attributes parameter isn't passed on. Good catch. I think the requests would be queued. So please modify the @return. Definitely. It would be nice to put trunk patch on review board. Its, unfortunately, going to be more than a little bit different. But yeah, I'll post one up soon
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching patch addressing Ted's comments and catching a couple hooks I missed the first time around.

          Show
          jesse_yates Jesse Yates added a comment - Attaching patch addressing Ted's comments and catching a couple hooks I missed the first time around.
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching trunk patch and on review board: https://reviews.apache.org/r/9312/

          Show
          jesse_yates Jesse Yates added a comment - Attaching trunk patch and on review board: https://reviews.apache.org/r/9312/
          Hide
          jesse_yates Jesse Yates added a comment -

          Updated trunk patch. I'd like to commit this soon (end of the week at the latest), unless there are any objections.

          Andrew Purtell?

          Show
          jesse_yates Jesse Yates added a comment - Updated trunk patch. I'd like to commit this soon (end of the week at the latest), unless there are any objections. Andrew Purtell ?
          Hide
          stack stack added a comment -

          Patch looks good to me. Mr Component Chief [~adp], what you reckon sir?

          Show
          stack stack added a comment - Patch looks good to me. Mr Component Chief [~adp] , what you reckon sir?
          Hide
          sershe Sergey Shelukhin added a comment -

          left some non-blocking comments in /r/. I am assuming Stack meant Andrew Purtell in the last comment?

          Show
          sershe Sergey Shelukhin added a comment - left some non-blocking comments in /r/. I am assuming Stack meant Andrew Purtell in the last comment?
          Hide
          apurtell Andrew Purtell added a comment -

          I think downcasting is a code smell and should be avoided unless necessary.

          This is not necessarily better: What about passing the CompactionRequest through the CP hooks and making it so the CompactionRequest type can be extended?

          Show
          apurtell Andrew Purtell added a comment - I think downcasting is a code smell and should be avoided unless necessary. This is not necessarily better: What about passing the CompactionRequest through the CP hooks and making it so the CompactionRequest type can be extended?
          Hide
          jesse_yates Jesse Yates added a comment - - edited

          Thanks Sergey Shelukhin! Since I've got +1s from stack and Ted (and it looks like Sergey Shelukhin blessing too?), I'm planning on committing v1 to trunk and 0.94 tonight, unless there are any objections.

          Show
          jesse_yates Jesse Yates added a comment - - edited Thanks Sergey Shelukhin ! Since I've got +1s from stack and Ted (and it looks like Sergey Shelukhin blessing too?), I'm planning on committing v1 to trunk and 0.94 tonight, unless there are any objections.
          Hide
          apurtell Andrew Purtell added a comment -

          Fine, -1.

          Consider the above. Then let's talk.

          Show
          apurtell Andrew Purtell added a comment - Fine, -1. Consider the above. Then let's talk.
          Hide
          jesse_yates Jesse Yates added a comment -

          I'd need to control a compaction request from inception to execution. Looking at the way we make compactions, there are the pre/postCompactSelection which would need to get the attributes. Then we could definitely add a hook at the end where we can override the compactionRequest:

          ...
          // everything went better than expected. create a compaction request
          int pri = getCompactPriority(priority);
          ret = new CompactionRequest(region, this, filesToCompact, isMajor, pri,
            compactionAttributes);
          ...
          

          to something like:

          ...
          // everything went better than expected. create a compaction request
          int pri = getCompactPriority(priority);
          if (region.getCoprocessorHost() != null) {
            ret = region.getCoprocessorHost().preCompactionRequest(region, this, filesToCompact, isMajor, pri, compactionAttributes)
          }
          if(ret == null){
          ret = new CompactionRequest(region, this, filesToCompact, isMajor, pri, compactionAttributes);
          }
          

          Then we'd just have to pass in the compaction request to all the compaction CP hooks, rather than the compactionAttributes object so we can handle special case compaction setup (for methods like preCompactScannerOpen and preCompact).

          So the same overall usage and layout, but we don't need to modify CompactionRequest and let people add their own finish hooks and store attributes if the want.

          I dig it Andrew. Sound reasonable to everyone else?

          Show
          jesse_yates Jesse Yates added a comment - I'd need to control a compaction request from inception to execution. Looking at the way we make compactions, there are the pre/postCompactSelection which would need to get the attributes. Then we could definitely add a hook at the end where we can override the compactionRequest: ... // everything went better than expected. create a compaction request int pri = getCompactPriority(priority); ret = new CompactionRequest(region, this , filesToCompact, isMajor, pri, compactionAttributes); ... to something like: ... // everything went better than expected. create a compaction request int pri = getCompactPriority(priority); if (region.getCoprocessorHost() != null ) { ret = region.getCoprocessorHost().preCompactionRequest(region, this , filesToCompact, isMajor, pri, compactionAttributes) } if (ret == null ){ ret = new CompactionRequest(region, this , filesToCompact, isMajor, pri, compactionAttributes); } Then we'd just have to pass in the compaction request to all the compaction CP hooks, rather than the compactionAttributes object so we can handle special case compaction setup (for methods like preCompactScannerOpen and preCompact). So the same overall usage and layout, but we don't need to modify CompactionRequest and let people add their own finish hooks and store attributes if the want. I dig it Andrew. Sound reasonable to everyone else?
          Hide
          apurtell Andrew Purtell added a comment -

          I do like it better, thanks for listening, +0.

          Show
          apurtell Andrew Purtell added a comment - I do like it better, thanks for listening, +0.
          Hide
          sershe Sergey Shelukhin added a comment -

          hmm, allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes.
          The latter is a common pattern when calling plugins that cannot preserve information across calls.

          Show
          sershe Sergey Shelukhin added a comment - hmm, allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes. The latter is a common pattern when calling plugins that cannot preserve information across calls.
          Hide
          sershe Sergey Shelukhin added a comment -

          Another alternative, uglier but with more general applicability, is to create some ID for the operation and pass it to coprocessor methods if they belong to the same logical operation. If coprocessor has its own context, it can manage it, for example in a Map.

          Show
          sershe Sergey Shelukhin added a comment - Another alternative, uglier but with more general applicability, is to create some ID for the operation and pass it to coprocessor methods if they belong to the same logical operation. If coprocessor has its own context, it can manage it, for example in a Map.
          Hide
          jesse_yates Jesse Yates added a comment -

          Another alternative, uglier but with more general applicability, is to create some ID for the operation and pass it to coprocessor methods if they belong to the same logical operation. If coprocessor has its own context, it can manage it, for example in a Map.

          This would be far better managed as a set of attributes attached to a CompactionRequest. Generally, you aren't going to start a compaction from one of the generic RegionObserver methods, so its going to be an Endpoint, which mean another class, so you then need to keep a static Map shared across the classes.

          allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes.

          Sure, but CPs are inherently more dangerous. They are there for you people able to futz around with the internals and come with no warranty In our use case, we will probably just subclass the exsiting CompactionRequest and add the things that I added in the attached patches (v0, v1). I think adding this subclass would be a reasonable inclusion, at the least in the hbase-examples module, if not in the standard directories.

          Show
          jesse_yates Jesse Yates added a comment - Another alternative, uglier but with more general applicability, is to create some ID for the operation and pass it to coprocessor methods if they belong to the same logical operation. If coprocessor has its own context, it can manage it, for example in a Map. This would be far better managed as a set of attributes attached to a CompactionRequest. Generally, you aren't going to start a compaction from one of the generic RegionObserver methods, so its going to be an Endpoint, which mean another class, so you then need to keep a static Map shared across the classes. allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes. Sure, but CPs are inherently more dangerous. They are there for you people able to futz around with the internals and come with no warranty In our use case, we will probably just subclass the exsiting CompactionRequest and add the things that I added in the attached patches (v0, v1). I think adding this subclass would be a reasonable inclusion, at the least in the hbase-examples module, if not in the standard directories.
          Hide
          apurtell Andrew Purtell added a comment - - edited

          allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes

          I see it as being more powerful, and a bit better structured. This is Coprocessors, dangerous is implied.

          Show
          apurtell Andrew Purtell added a comment - - edited allowing the user to create compaction requests seems more dangerous to me than having an opaque context/attributes I see it as being more powerful, and a bit better structured. This is Coprocessors, dangerous is implied.
          Hide
          sershe Sergey Shelukhin added a comment -

          Hmm, sure. +1. CompactionRequest then also needs InterfaceAudience, and ctor cannot be removed later.

          Show
          sershe Sergey Shelukhin added a comment - Hmm, sure. +1. CompactionRequest then also needs InterfaceAudience, and ctor cannot be removed later.
          Hide
          jesse_yates Jesse Yates added a comment -

          Not necessarily - we already have a couple cases where we override an InterfaceAudience.Private. See RegionCoprocessorHost# postScannerOpen(final Scan scan, RegionScanner s), which returns a RegionScanner.

          Show
          jesse_yates Jesse Yates added a comment - Not necessarily - we already have a couple cases where we override an InterfaceAudience.Private. See RegionCoprocessorHost# postScannerOpen(final Scan scan, RegionScanner s), which returns a RegionScanner.
          Hide
          apurtell Andrew Purtell added a comment -

          We don't have consensus on if interface annotations are applicable to coprocessors. I'd be cool with LimitedPrivate(

          {"coprocessors"}

          ) and Evolving.

          Show
          apurtell Andrew Purtell added a comment - We don't have consensus on if interface annotations are applicable to coprocessors. I'd be cool with LimitedPrivate( {"coprocessors"} ) and Evolving.
          Hide
          sershe Sergey Shelukhin added a comment -

          So, is there consensus on letting coproc to subclass CompactionRequest? I'm curious to look at the code change involved, for some other purpose

          Show
          sershe Sergey Shelukhin added a comment - So, is there consensus on letting coproc to subclass CompactionRequest? I'm curious to look at the code change involved, for some other purpose
          Hide
          jesse_yates Jesse Yates added a comment -

          That's what I was thinking - posted an update to RB with what the refactor looks like.

          Show
          jesse_yates Jesse Yates added a comment - That's what I was thinking - posted an update to RB with what the refactor looks like.
          Hide
          apurtell Andrew Purtell added a comment -

          So, is there consensus on letting coproc to subclass CompactionRequest

          +1

          Show
          apurtell Andrew Purtell added a comment - So, is there consensus on letting coproc to subclass CompactionRequest +1
          Hide
          sershe Sergey Shelukhin added a comment -

          posted responses on /r/, btw

          Show
          sershe Sergey Shelukhin added a comment - posted responses on /r/, btw
          Hide
          sershe Sergey Shelukhin added a comment -

          Jesse Yates do you want to commit soonish? I already made some cleanup of compaction request that can cause minor merge, bigger changes are afoot, just want to avoid big conflicts. Thanks!

          Show
          sershe Sergey Shelukhin added a comment - Jesse Yates do you want to commit soonish? I already made some cleanup of compaction request that can cause minor merge, bigger changes are afoot, just want to avoid big conflicts. Thanks!
          Hide
          jesse_yates Jesse Yates added a comment -

          Yeah, I'll get on it. got a little sidetracked at the office, but I'll try to post the updated version on RB tonight/tomorrow am. Thanks for the heads up.

          Show
          jesse_yates Jesse Yates added a comment - Yeah, I'll get on it. got a little sidetracked at the office, but I'll try to post the updated version on RB tonight/tomorrow am. Thanks for the heads up.
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching updated patch (on trunk + Sergey's comments). Also posted on RB.

          Show
          jesse_yates Jesse Yates added a comment - Attaching updated patch (on trunk + Sergey's comments). Also posted on RB.
          Hide
          sershe Sergey Shelukhin added a comment -

          I don't think I understand why we have to make attributes visible to HBase code.
          For example, when we call this:
          server.compactSplitThread.requestCompaction(region, store, "Recursive enqueue", attributes);
          we are inside old CompactionRequest, so we could just pass "this" instead of attributes all the way down, and coproc would get attributes if it needs to. That way HBase code doesn't even have to know attributes exist inside coproc-subclasses CompactionRequest.
          And coproc has to cast in this case anyway, either CR to subclass, or Object to specific attribute type.
          But, this shouldn't be blocking, so +1

          Show
          sershe Sergey Shelukhin added a comment - I don't think I understand why we have to make attributes visible to HBase code. For example, when we call this: server.compactSplitThread.requestCompaction(region, store, "Recursive enqueue", attributes); we are inside old CompactionRequest, so we could just pass "this" instead of attributes all the way down, and coproc would get attributes if it needs to. That way HBase code doesn't even have to know attributes exist inside coproc-subclasses CompactionRequest. And coproc has to cast in this case anyway, either CR to subclass, or Object to specific attribute type. But, this shouldn't be blocking, so +1
          Hide
          jesse_yates Jesse Yates added a comment -

          I don't think I understand why we have to make attributes visible to HBase code.
          For example, when we call this:
          server.compactSplitThread.requestCompaction(region, store, "Recursive enqueue", attributes);
          we are inside old CompactionRequest, so we could just pass "this" instead of attributes all the way down, and coproc would get attributes if it needs to. That way HBase code doesn't even have to know attributes exist inside coproc-subclasses CompactionRequest.

          The question to me came down to be whether the re-enque is supposed to be a 'new' compaction request or a re-attempt at the same compaction. In the existing code, its a brand new compaction request that just happens to be on the same region and store.

          So, do we pass along the original attributes or do we make it a 'system created' compaction, separate from the original request?

          My inclination was to pass the same attributes back along again. Therefore, for example, the CP that creates the CompactionRequest can check the attributes and make a decision as to whether or not it should modify the compaction ("Oh, I already made this request! That means I need to ..."). Likely, you aren't going to want to do anything again, but this is a bit more logic and knowledge of the compaction architecture than most people are going to want to deal with ("What, why am I getting this request again! The one I ran is done!"). But this is CPs and you are supposed to have the rope to hang yourself

          The flip side is that we consider this a 'system' compaction separate from the user requested one and just pass in null attributes. This is nice in that its cleaner (no leaked attributes) and logically is a different operation from the original one that that client started. For instance, suppose the client starts a compaction, but their compaction doesn't do actually change the number of files. If we have a blocking number of store files we could re-enque that compaction as a regular system operation.

          Thoughts? I could go either way.

          Show
          jesse_yates Jesse Yates added a comment - I don't think I understand why we have to make attributes visible to HBase code. For example, when we call this: server.compactSplitThread.requestCompaction(region, store, "Recursive enqueue", attributes); we are inside old CompactionRequest, so we could just pass "this" instead of attributes all the way down, and coproc would get attributes if it needs to. That way HBase code doesn't even have to know attributes exist inside coproc-subclasses CompactionRequest. The question to me came down to be whether the re-enque is supposed to be a 'new' compaction request or a re-attempt at the same compaction. In the existing code, its a brand new compaction request that just happens to be on the same region and store. So, do we pass along the original attributes or do we make it a 'system created' compaction, separate from the original request? My inclination was to pass the same attributes back along again. Therefore, for example, the CP that creates the CompactionRequest can check the attributes and make a decision as to whether or not it should modify the compaction ("Oh, I already made this request! That means I need to ..."). Likely, you aren't going to want to do anything again, but this is a bit more logic and knowledge of the compaction architecture than most people are going to want to deal with ("What, why am I getting this request again! The one I ran is done!"). But this is CPs and you are supposed to have the rope to hang yourself The flip side is that we consider this a 'system' compaction separate from the user requested one and just pass in null attributes. This is nice in that its cleaner (no leaked attributes) and logically is a different operation from the original one that that client started. For instance, suppose the client starts a compaction, but their compaction doesn't do actually change the number of files. If we have a blocking number of store files we could re-enque that compaction as a regular system operation. Thoughts? I could go either way.
          Hide
          apurtell Andrew Purtell added a comment -

          Therefore, for example, the CP that creates the CompactionRequest can check the attributes and make a decision as to whether or not it should modify the compaction ("Oh, I already made this request! That means I need to ...").

          Or define a Comparator for CompactionRequest and have the CP extend that too as needed when extending CR?

          Should not need generic Object attributes as parameter to preCompactRequest and may not need generic Object attributes to pre/postCompactSelection if we pass in the nascent CompactionRequest reference there too, but no strong feeling about this either way.

          Show
          apurtell Andrew Purtell added a comment - Therefore, for example, the CP that creates the CompactionRequest can check the attributes and make a decision as to whether or not it should modify the compaction ("Oh, I already made this request! That means I need to ..."). Or define a Comparator for CompactionRequest and have the CP extend that too as needed when extending CR? Should not need generic Object attributes as parameter to preCompactRequest and may not need generic Object attributes to pre/postCompactSelection if we pass in the nascent CompactionRequest reference there too, but no strong feeling about this either way.
          Hide
          apurtell Andrew Purtell added a comment -

          The flip side is that we consider this a 'system' compaction separate from the user requested one and just pass in null attributes.

          I do think you should avoid having to make this distinction.

          Show
          apurtell Andrew Purtell added a comment - The flip side is that we consider this a 'system' compaction separate from the user requested one and just pass in null attributes. I do think you should avoid having to make this distinction.
          Hide
          jesse_yates Jesse Yates added a comment -

          may not need generic Object attributes to pre/postCompactSelection in if we pass in the nascent CompactionRequest reference there too

          I worry that we will start to pervert the compactionRequest creation logic for what is a relatively rare case - a user requested compaction via a CP (or at least that was the original intent, not sure what everyone will do with this).

          But an interesting idea (and another refactor . So instead of passing in the attributes you pass in a compaction request to the CompactSplitThread? Something like, "if the request is null we do the usual logic and build the request only after we get through all the selection logic in HStore. If not null, we apply the same logic, but set attributes on the compaction request (e.g. the store files to compact) on the passed in request"? That certainly gets me what I need (controlled compactions) and hides the attribute business all the way down.

          Is that what you were getting at? I like it Andrew.

          Show
          jesse_yates Jesse Yates added a comment - may not need generic Object attributes to pre/postCompactSelection in if we pass in the nascent CompactionRequest reference there too I worry that we will start to pervert the compactionRequest creation logic for what is a relatively rare case - a user requested compaction via a CP (or at least that was the original intent, not sure what everyone will do with this). But an interesting idea (and another refactor . So instead of passing in the attributes you pass in a compaction request to the CompactSplitThread? Something like, "if the request is null we do the usual logic and build the request only after we get through all the selection logic in HStore. If not null, we apply the same logic, but set attributes on the compaction request (e.g. the store files to compact) on the passed in request"? That certainly gets me what I need (controlled compactions) and hides the attribute business all the way down. Is that what you were getting at? I like it Andrew.
          Hide
          jesse_yates Jesse Yates added a comment -

          I do think you should avoid having to make this distinction.

          Why? If we initiate a compaction and it does some stuff and then the system needs to go about doing its usual compaction business (its still has too many store files) then it ought to be able to. If you want to stop it, then you can hook into the usual compaction request creation mechansim. Am I missing something?

          Show
          jesse_yates Jesse Yates added a comment - I do think you should avoid having to make this distinction. Why? If we initiate a compaction and it does some stuff and then the system needs to go about doing its usual compaction business (its still has too many store files) then it ought to be able to. If you want to stop it, then you can hook into the usual compaction request creation mechansim. Am I missing something?
          Hide
          apurtell Andrew Purtell added a comment -

          That certainly gets me what I need (controlled compactions) and hides the attribute business all the way down. Is that what you were getting at?

          Yes.

          I do think you should avoid having to make this distinction.

          Why?

          I mean the (minor) extra cognitive load of figuring out if a CR is a system CR or user CR or why the difference.

          Show
          apurtell Andrew Purtell added a comment - That certainly gets me what I need (controlled compactions) and hides the attribute business all the way down. Is that what you were getting at? Yes. I do think you should avoid having to make this distinction. Why? I mean the (minor) extra cognitive load of figuring out if a CR is a system CR or user CR or why the difference.
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching updated patch based on Andrew's comments. Also posted to RB.

          Show
          jesse_yates Jesse Yates added a comment - Attaching updated patch based on Andrew's comments. Also posted to RB.
          Hide
          sershe Sergey Shelukhin added a comment -

          +1

          Show
          sershe Sergey Shelukhin added a comment - +1
          Hide
          apurtell Andrew Purtell added a comment -

          +1 patch looks good. Minor nit for on commit is the javadoc: "custom compaction request" -> "compaction request". May or may not be custom, right?

          Show
          apurtell Andrew Purtell added a comment - +1 patch looks good. Minor nit for on commit is the javadoc: "custom compaction request" -> "compaction request". May or may not be custom, right?
          Hide
          jesse_yates Jesse Yates added a comment -

          Minor nit for on commit is the javadoc: "custom compaction request" -> "compaction request". May or may not be custom, right?

          Yeah, could also be null for pre/postCompactSelection.

          Show
          jesse_yates Jesse Yates added a comment - Minor nit for on commit is the javadoc: "custom compaction request" -> "compaction request". May or may not be custom, right? Yeah, could also be null for pre/postCompactSelection.
          Hide
          jesse_yates Jesse Yates added a comment -

          I'd like to commit soon (this weekend) to get Sergey rolling on his compaction work, unless there are objections...?

          Show
          jesse_yates Jesse Yates added a comment - I'd like to commit soon (this weekend) to get Sergey rolling on his compaction work, unless there are objections...?
          Hide
          lhofhansl Lars Hofhansl added a comment -

          +1

          Wanna a simple (hopefully) test that tracks a CompactionRequest through the queue?

          Show
          lhofhansl Lars Hofhansl added a comment - +1 Wanna a simple (hopefully) test that tracks a CompactionRequest through the queue?
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching patch with test (thanks for the suggestion Lars Hofhansl!). Had to do a little patch to HRegionServer so it could be mocked out in the test - wrapping the direct access to the compactSplitThread in a getter (3 lines).

          I'd like to commit on Tuesday.

          Show
          jesse_yates Jesse Yates added a comment - Attaching patch with test (thanks for the suggestion Lars Hofhansl !). Had to do a little patch to HRegionServer so it could be mocked out in the test - wrapping the direct access to the compactSplitThread in a getter (3 lines). I'd like to commit on Tuesday.
          Hide
          apurtell Andrew Purtell added a comment -

          This looks good.

          +1 if the new test passes

          Show
          apurtell Andrew Purtell added a comment - This looks good. +1 if the new test passes
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12569835/hbase-7725-v5.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 6 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
          org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk
          org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
          org.apache.hadoop.hbase.backup.TestHFileArchiving
          org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
          org.apache.hadoop.hbase.regionserver.TestCompactionState
          org.apache.hadoop.hbase.client.TestShell
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
          org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction
          org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
          org.apache.hadoop.hbase.client.TestFromClientSide
          org.apache.hadoop.hbase.client.TestFromClientSide3

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12569835/hbase-7725-v5.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 6 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk org.apache.hadoop.hbase.io.encoding.TestChangingEncoding org.apache.hadoop.hbase.backup.TestHFileArchiving org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor org.apache.hadoop.hbase.regionserver.TestCompactionState org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.client.TestFromClientSide3 Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4455//console This message is automatically generated.
          Hide
          apurtell Andrew Purtell added a comment -

          -1 Some NPEs out of compaction code:

          Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException
          	at org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompaction(CompactSplitThread.java:217)
          	at org.apache.hadoop.hbase.regionserver.HRegionServer.compactRegion(HRegionServer.java:3633)
          
          Show
          apurtell Andrew Purtell added a comment - -1 Some NPEs out of compaction code: Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException: java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.CompactSplitThread.requestCompaction(CompactSplitThread.java:217) at org.apache.hadoop.hbase.regionserver.HRegionServer.compactRegion(HRegionServer.java:3633)
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching patch with the fix (Forgot an else and just dropped through the if :-/).

          Failed tests passed locally. Lets try this again...

          Show
          jesse_yates Jesse Yates added a comment - Attaching patch with the fix (Forgot an else and just dropped through the if :-/). Failed tests passed locally. Lets try this again...
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12569873/hbase-7725-v6.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          -1 javadoc. The javadoc tool appears to have generated 6 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestHLog

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12569873/hbase-7725-v6.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 6 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLog Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4457//console This message is automatically generated.
          Hide
          jesse_yates Jesse Yates added a comment -

          Looks like the test failures are unrelated and it passes locally.

          Show
          jesse_yates Jesse Yates added a comment - Looks like the test failures are unrelated and it passes locally.
          Hide
          apurtell Andrew Purtell added a comment -

          The test failures are due to this:

          java.net.BindException: Problem binding to localhost/127.0.0.1:42113 : Address already in use
          

          +1 for commit, thanks Jesse!

          Show
          apurtell Andrew Purtell added a comment - The test failures are due to this: java.net.BindException: Problem binding to localhost/127.0.0.1:42113 : Address already in use +1 for commit, thanks Jesse!
          Hide
          jesse_yates Jesse Yates added a comment -

          Cool, thanks Andy! I'm planning to commmit this evening then, unless there are objections.

          Show
          jesse_yates Jesse Yates added a comment - Cool, thanks Andy! I'm planning to commmit this evening then, unless there are objections.
          Hide
          jesse_yates Jesse Yates added a comment -

          Submitted to trunk. Working on a patch for 0.94

          Show
          jesse_yates Jesse Yates added a comment - Submitted to trunk. Working on a patch for 0.94
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Jesse: the following javadoc warnings may have been caused by your checkin.

          [WARNING] Javadoc Warnings
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java:51: warning - @param argument "filesToCompact" is not a parameter name.
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java:51: warning - Tag @link: can't find compact(Collection, boolean) in org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:178: warning - Tag @link: can't find postCompactSelection(ObserverContext, Store, ImmutableList, Object) in org.apache.hadoop.hbase.coprocessor.RegionObserver
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:153: warning - Tag @link: can't find preCompactSelection(ObserverContext, Store, List, Object) in org.apache.hadoop.hbase.coprocessor.RegionObserver
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:178: warning - @param argument "compactionAttributes" is not a parameter name.
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:233: warning - @param argument "request" is not a parameter name.
          [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:283: warning - @param argument "request" is not a parameter name.
          

          Can you attach an addendum to fix them ?

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Jesse: the following javadoc warnings may have been caused by your checkin. [WARNING] Javadoc Warnings [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java:51: warning - @param argument "filesToCompact" is not a parameter name. [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java:51: warning - Tag @link: can't find compact(Collection, boolean ) in org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:178: warning - Tag @link: can't find postCompactSelection(ObserverContext, Store, ImmutableList, Object ) in org.apache.hadoop.hbase.coprocessor.RegionObserver [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:153: warning - Tag @link: can't find preCompactSelection(ObserverContext, Store, List, Object ) in org.apache.hadoop.hbase.coprocessor.RegionObserver [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:178: warning - @param argument "compactionAttributes" is not a parameter name. [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:233: warning - @param argument "request" is not a parameter name. [WARNING] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java:283: warning - @param argument "request" is not a parameter name. Can you attach an addendum to fix them ?
          Hide
          jesse_yates Jesse Yates added a comment -

          Oh, yeah. thanks Ted - getting right on it.

          Show
          jesse_yates Jesse Yates added a comment - Oh, yeah. thanks Ted - getting right on it.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          While you're composing addendum, do you mind wrapping long lines ?

          +   *             {@link #preCompact(ObserverContext, Store, InternalScanner, ScanType, CompactionRequest)}
          ...
          +   *             {@link #preCompactScannerOpen(ObserverContext, Store, List, ScanType, long, InternalScanner, CompactionRequest)}
          

          Thanks

          Show
          yuzhihong@gmail.com Ted Yu added a comment - While you're composing addendum, do you mind wrapping long lines ? + * {@link #preCompact(ObserverContext, Store, InternalScanner, ScanType, CompactionRequest)} ... + * {@link #preCompactScannerOpen(ObserverContext, Store, List, ScanType, long , InternalScanner, CompactionRequest)} Thanks
          Hide
          jesse_yates Jesse Yates added a comment - - edited

          They don't wrap naturally - the method name should all be the same line.

          Show
          jesse_yates Jesse Yates added a comment - - edited They don't wrap naturally - the method name should all be the same line.
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK #3887 (See https://builds.apache.org/job/HBase-TRUNK/3887/)
          HBASE-7725: Add ability to create custom compaction request (Revision 1448449)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK #3887 (See https://builds.apache.org/job/HBase-TRUNK/3887/ ) HBASE-7725 : Add ability to create custom compaction request (Revision 1448449) Result = FAILURE jyates : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #413 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/413/)
          HBASE-7725: Add ability to create custom compaction request (Revision 1448449)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #413 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/413/ ) HBASE-7725 : Add ability to create custom compaction request (Revision 1448449) Result = FAILURE jyates : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching addendum for fixing the javadoc issues + line size. thanks for the catch Ted Yu!

          Show
          jesse_yates Jesse Yates added a comment - Attaching addendum for fixing the javadoc issues + line size. thanks for the catch Ted Yu !
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570269/hbase-7725-addenum.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces lines longer than 100

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570269/hbase-7725-addenum.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces lines longer than 100 +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4481//console This message is automatically generated.
          Hide
          jesse_yates Jesse Yates added a comment -

          Fat fingered eclipse - lets try with (hopefully) a better addendum

          Show
          jesse_yates Jesse Yates added a comment - Fat fingered eclipse - lets try with (hopefully) a better addendum
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570324/hbase-7725-addenum-v1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570324/hbase-7725-addenum-v1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4483//console This message is automatically generated.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          +1 on addendum.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - +1 on addendum.
          Hide
          jesse_yates Jesse Yates added a comment -

          Committed addendum to trunk.

          Show
          jesse_yates Jesse Yates added a comment - Committed addendum to trunk.
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK #3889 (See https://builds.apache.org/job/HBase-TRUNK/3889/)
          HBASE-7725: Add ability to create custom compaction request - ADDENDUM: fix javadocs (Revision 1448778)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK #3889 (See https://builds.apache.org/job/HBase-TRUNK/3889/ ) HBASE-7725 : Add ability to create custom compaction request - ADDENDUM: fix javadocs (Revision 1448778) Result = FAILURE jyates : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #415 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/415/)
          HBASE-7725: Add ability to create custom compaction request - ADDENDUM: fix javadocs (Revision 1448778)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          Show
          hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #415 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/415/ ) HBASE-7725 : Add ability to create custom compaction request - ADDENDUM: fix javadocs (Revision 1448778) Result = FAILURE jyates : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/DefaultCompactor.java
          Hide
          jesse_yates Jesse Yates added a comment -

          Attaching patch for 0.94 based on the trunk version. TestCompaction and the above tests seem to pass locally.

          I'd like to commit on Wednesday (3/27) unless there are objections.

          Show
          jesse_yates Jesse Yates added a comment - Attaching patch for 0.94 based on the trunk version. TestCompaction and the above tests seem to pass locally. I'd like to commit on Wednesday (3/27) unless there are objections.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12570912/hbase-7725_0.94-v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4546//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570912/hbase-7725_0.94-v3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4546//console This message is automatically generated.
          Hide
          lhofhansl Lars Hofhansl added a comment -

          +1 on 0.94 patch.

          Show
          lhofhansl Lars Hofhansl added a comment - +1 on 0.94 patch.
          Hide
          jesse_yates Jesse Yates added a comment -

          Updating patch with slight bug fix (added to the wrong list in one of the CompactSplitThread#requestCompaction) and adding a test to make sure it doesn't happen again.

          Filed HBASE-7947 to add the same test to 0.95/trunk

          Show
          jesse_yates Jesse Yates added a comment - Updating patch with slight bug fix (added to the wrong list in one of the CompactSplitThread#requestCompaction) and adding a test to make sure it doesn't happen again. Filed HBASE-7947 to add the same test to 0.95/trunk
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12571100/hbase-7725_0.94-v4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4564//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571100/hbase-7725_0.94-v4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4564//console This message is automatically generated.
          Hide
          jesse_yates Jesse Yates added a comment -

          Committed to 0.94.6 - closing issue.

          Show
          jesse_yates Jesse Yates added a comment - Committed to 0.94.6 - closing issue.
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94 #868 (See https://builds.apache.org/job/HBase-0.94/868/)
          HBASE-7725: Add ability to create custom compaction request (Revision 1451057)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94 #868 (See https://builds.apache.org/job/HBase-0.94/868/ ) HBASE-7725 : Add ability to create custom compaction request (Revision 1451057) Result = FAILURE jyates : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94-security #113 (See https://builds.apache.org/job/HBase-0.94-security/113/)
          HBASE-7725: Add ability to create custom compaction request (Revision 1451057)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94-security #113 (See https://builds.apache.org/job/HBase-0.94-security/113/ ) HBASE-7725 : Add ability to create custom compaction request (Revision 1451057) Result = FAILURE jyates : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          hudson Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #12 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/12/)
          HBASE-7725: Add ability to create custom compaction request (Revision 1451057)

          Result = FAILURE
          jyates :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          hudson Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #12 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/12/ ) HBASE-7725 : Add ability to create custom compaction request (Revision 1451057) Result = FAILURE jyates : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java

            People

            • Assignee:
              jesse_yates Jesse Yates
              Reporter:
              jesse_yates Jesse Yates
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development