Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Later
    • Affects Version/s: 0.90.2, 0.90.3, 0.90.4, 0.92.0
    • Fix Version/s: None
    • Component/s: master, regionserver
    • Labels:
      None
    • Release Note:
      Hide
      Patch used for table priority alone,In this stand alone patch, not only tables can have different priorities but also the different actions like "get","scan","put" and "delete" can have priorities.
      Show
      Patch used for table priority alone,In this stand alone patch, not only tables can have different priorities but also the different actions like "get","scan","put" and "delete" can have priorities.

      Description

      The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables.
      When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished.
      After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group.
      In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables
      Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size.
      Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster.

      git entry : https://github.com/ICT-Ope/HBase_allocation .

      We hope our work is helpful.

        Attachments

        1. TablePriority_v17.patch
          116 kB
          Liu Jia
        2. TablePriority_v17.patch
          116 kB
          Liu Jia
        3. Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf
          494 kB
          Liu Jia
        4. TablePriority_v16_with_coprocessor.patch
          82 kB
          Liu Jia
        5. TablePriority_v15_with_coprocessor.patch
          79 kB
          Liu Jia
        6. TablePriority_v12.patch
          78 kB
          Liu Jia
        7. TablePriority_v12.patch
          78 kB
          Liu Jia
        8. TablePrioriy_v9.patch
          76 kB
          Liu Jia
        9. TablePriority_v8.patch
          76 kB
          Liu Jia
        10. TablePriority_v8.patch
          74 kB
          Ted Yu
        11. TablePriority_v8_for_trunk.patch
          74 kB
          Liu Jia
        12. TablePriority.patch
          37 kB
          Liu Jia
        13. Design_document_for_HBase_isolation_and_allocation_Revised.pdf
          558 kB
          Liu Jia
        14. System Structure.jpg
          61 kB
          Liu Jia
        15. Design_document_for_HBase_isolation_and_allocation.pdf
          140 kB
          Liu Jia
        16. HBase_isolation_and_allocation_user_guide.pdf
          1.77 MB
          Liu Jia
        17. Performance_of_Table_priority.pdf
          486 kB
          Liu Jia

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                liujia_ict Liu Jia
              • Votes:
                3 Vote for this issue
                Watchers:
                38 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 504h
                  504h
                  Remaining:
                  Remaining Estimate - 504h
                  504h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified