HBase
  1. HBase
  2. HBASE-6091

Come up with strawman proposal for RC testing matrix

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Implemented
    • Affects Version/s: 0.95.2
    • Fix Version/s: None
    • Component/s: documentation
    • Labels:
      None

      Activity

      Hide
      David S. Wang added a comment -

      I have some content ready; however is there some format I should post it in (e.g. Confluence wiki, etc.)?

      Show
      David S. Wang added a comment - I have some content ready; however is there some format I should post it in (e.g. Confluence wiki, etc.)?
      Hide
      stack added a comment -

      This seems like good candidate for wiki page at least until it hardens some. Thereafter, it could go into the reference guide? Good on you David.

      Show
      stack added a comment - This seems like good candidate for wiki page at least until it hardens some. Thereafter, it could go into the reference guide? Good on you David.
      Hide
      David S. Wang added a comment -

      I signed up for a wiki page account, but cannot seem to create a new page, or at least I could not figure out how to do that. Can I get perms to create and edit my own page on the hbase wiki? Otherwise I can just attach what I have so far in ugly text format.

      Show
      David S. Wang added a comment - I signed up for a wiki page account, but cannot seem to create a new page, or at least I could not figure out how to do that. Can I get perms to create and edit my own page on the hbase wiki? Otherwise I can just attach what I have so far in ugly text format.
      Hide
      David S. Wang added a comment -

      Here's a first draft in Atlassian JIRA format. I can convert this to something else if needed.

      Release version: (e.g. 0.94.1 RC2)

      Category Reporter Underlying Hadoop version Tests run +1/0/-1 Results details
      Master
      Log splitting Dave Hadoop 2.0.0 e.g. Configured no distributed log splitting, killed RS +1 e.g. looked at logs, verified no data loss
      Distributed log splitting          
      Load balancer          
      Regions in transition   Hadoop 1.0.3 e.g. unassign region(s) from shell, see that they get reassigned after a while   e.g. check logs that region was in transition for a little while, not stuck there
      Region server
      Flushes          
      Compactions          
      Splits          
      Block cache          
      Client
      Java          
      Thrift          
      REST          
      Shell          
      Tools
      scripts (e.g. graceful stop)          
      Bulk load          
      CopyTable          
      importtsv          
      hbck          
      Metrics
      Coprocessors
      Security
      Kerberos authentication          
      ACL coprocessor          
      TokenProvider coprocessor          
      Replication
      Master-slave          
      Master-master          
      Cyclic          
      Compression
      Snappy          
      LZO          
      Performance
      YCSB          
      PerformanceEvaluation          
      Compatibility
      Client-server compatibility between minor versions          
      Server-server compatibility between minor versions          
      Client-server compatibility between major versions -+1          
      Server-server compatibility between major versions -+1          
      Client-server compatibility between major versions > -+1          
      Server-server compatibility between major versions > -+1          
      Manual rolling restart between this version and previous version (minor or major)          
      Configuration
      System
      TestLoadAndVerify          
      TestAcidGuarantees          
      TestConcurrentScanAndPut          
      Failover to backup master          
      Kill RS          
      Kill RS holding .META.          
      Kill RS holding ROOT          
      New features
      e.g. Snapshots          
      Documentation
      Review section <X> of reference guide          
      Review bundled documentation          
      Show
      David S. Wang added a comment - Here's a first draft in Atlassian JIRA format. I can convert this to something else if needed. Release version: (e.g. 0.94.1 RC2) Category Reporter Underlying Hadoop version Tests run +1/0/-1 Results details Master Log splitting Dave Hadoop 2.0.0 e.g. Configured no distributed log splitting, killed RS +1 e.g. looked at logs, verified no data loss Distributed log splitting           Load balancer           Regions in transition   Hadoop 1.0.3 e.g. unassign region(s) from shell, see that they get reassigned after a while   e.g. check logs that region was in transition for a little while, not stuck there Region server Flushes           Compactions           Splits           Block cache           Client Java           Thrift           REST           Shell           Tools scripts (e.g. graceful stop)           Bulk load           CopyTable           importtsv           hbck           Metrics Coprocessors Security Kerberos authentication           ACL coprocessor           TokenProvider coprocessor           Replication Master-slave           Master-master           Cyclic           Compression Snappy           LZO           Performance YCSB           PerformanceEvaluation           Compatibility Client-server compatibility between minor versions           Server-server compatibility between minor versions           Client-server compatibility between major versions -+1           Server-server compatibility between major versions -+1           Client-server compatibility between major versions > -+1           Server-server compatibility between major versions > -+1           Manual rolling restart between this version and previous version (minor or major)           Configuration System TestLoadAndVerify           TestAcidGuarantees           TestConcurrentScanAndPut           Failover to backup master           Kill RS           Kill RS holding .META.           Kill RS holding ROOT           New features e.g. Snapshots           Documentation Review section <X> of reference guide           Review bundled documentation          
      Hide
      stack added a comment -

      @David Thats ruhl phrutty I think the formatting on hbase/hadoop wiki the same. Whats your wiki name so I can give you access? Thanks.

      Show
      stack added a comment - @David Thats ruhl phrutty I think the formatting on hbase/hadoop wiki the same. Whats your wiki name so I can give you access? Thanks.
      Hide
      David S. Wang added a comment -

      My wiki name is DavidWang.

      Show
      David S. Wang added a comment - My wiki name is DavidWang.
      Hide
      stack added a comment -

      Try it now boss. Let me know if it doesn't work. Hopefully you can just copy paste the beauty you made above.

      Show
      stack added a comment - Try it now boss. Let me know if it doesn't work. Hopefully you can just copy paste the beauty you made above.
      Hide
      David S. Wang added a comment -

      I posted it at:

      http://wiki.apache.org/hadoop/ReleaseCandidateTestMatrix

      and made a link to it from:

      http://wiki.apache.org/hadoop/Hbase/HowToRelease

      I had to edit the source a bit as the Apache JIRA is not Atlassian format.

      What are the next steps here?

      Show
      David S. Wang added a comment - I posted it at: http://wiki.apache.org/hadoop/ReleaseCandidateTestMatrix and made a link to it from: http://wiki.apache.org/hadoop/Hbase/HowToRelease I had to edit the source a bit as the Apache JIRA is not Atlassian format. What are the next steps here?
      Hide
      stack added a comment -

      Tell dev list it exists. Chase us if we don't make an attempt at using the template whenever we go to make a release. Thanks David.

      Show
      stack added a comment - Tell dev list it exists. Chase us if we don't make an attempt at using the template whenever we go to make a release. Thanks David.
      Hide
      Andrew Purtell added a comment -

      Under the performance section of http://wiki.apache.org/hadoop/ReleaseCandidateTestMatrix, we have YCSB and PerformanceEvaluation. One is a tool in our tree that we can maintain, the other is not. I'll note we have LoadTestTool now from 0.92 forward. Why don't we use that instead of YCSB?

      Show
      Andrew Purtell added a comment - Under the performance section of http://wiki.apache.org/hadoop/ReleaseCandidateTestMatrix , we have YCSB and PerformanceEvaluation. One is a tool in our tree that we can maintain, the other is not. I'll note we have LoadTestTool now from 0.92 forward. Why don't we use that instead of YCSB?
      Hide
      David S. Wang added a comment -

      YCSB is something we use quite a bit and is easy to setup. I don't see the lack of ownership of the tool as something that would preclude it from being used at least for some basic first-level testing. Why don't we have both? I've added LoadTestTool to the matrix.

      Show
      David S. Wang added a comment - YCSB is something we use quite a bit and is easy to setup. I don't see the lack of ownership of the tool as something that would preclude it from being used at least for some basic first-level testing. Why don't we have both? I've added LoadTestTool to the matrix.
      Hide
      Andrew Purtell added a comment - - edited

      I've not used YCSB [edit: except in a limited way out of curiosity] but if I recall correctly Ryan Rawson and others had to rework it's HBase client code substantially. You using that version?

      Agree it depends what we are after. I think first order of approximation initial testing is covered by PerformanceEvaluation. Tools like YCSB or LoadTestTool I'd expect to more accurately measure something. It's likely the methodology for such measurements will be imperfect at the start and will need to evolve as HBase evolves. Therefore the tool should evolve in tandem.

      Show
      Andrew Purtell added a comment - - edited I've not used YCSB [edit: except in a limited way out of curiosity] but if I recall correctly Ryan Rawson and others had to rework it's HBase client code substantially. You using that version? Agree it depends what we are after. I think first order of approximation initial testing is covered by PerformanceEvaluation. Tools like YCSB or LoadTestTool I'd expect to more accurately measure something. It's likely the methodology for such measurements will be imperfect at the start and will need to evolve as HBase evolves. Therefore the tool should evolve in tandem.
      Hide
      David S. Wang added a comment -

      Probably ... IIRC we are using YCSB 0.14.

      In any case, it's not like every single field in the matrix has to be filled out before an RC is +1'ed; that would make RCs take a very long time indeed. I think leaving them both in is good for now; if it turns out that some rows never get filled out, we can always remove them later.

      Show
      David S. Wang added a comment - Probably ... IIRC we are using YCSB 0.14. In any case, it's not like every single field in the matrix has to be filled out before an RC is +1'ed; that would make RCs take a very long time indeed. I think leaving them both in is good for now; if it turns out that some rows never get filled out, we can always remove them later.

        People

        • Assignee:
          David S. Wang
          Reporter:
          David S. Wang
        • Votes:
          0 Vote for this issue
          Watchers:
          9 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development