HBase
  1. HBase
  2. HBASE-3647

Distinguish read and write request count in region

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Distinguishing read and write request counts, on top of HBASE-3507, would benefit load balancer.
      The action for balancing read vs. write load should be different. For read load, region movement should be low (to keep scanner happy). For write load, region movement is allowed.
      Now that we have cheap(er) counters, it should not be too burdensome keeping up the extra count.

      1. 3647-versioned.txt
        19 kB
        Ted Yu
      2. 3647-versioned-2.txt
        19 kB
        Ted Yu
      3. hbase-3647.txt
        17 kB
        Ted Yu

        Issue Links

          Activity

          Hide
          Ted Yu added a comment -

          Also includes the change of HServerLoad.regionLoad to Map.

          Show
          Ted Yu added a comment - Also includes the change of HServerLoad.regionLoad to Map.
          Hide
          Ted Yu added a comment -

          Removed unrelated change in HServerLoad

          Show
          Ted Yu added a comment - Removed unrelated change in HServerLoad
          Hide
          Ted Yu added a comment -

          Changed types of read/writeRequestsCount to int.

          Show
          Ted Yu added a comment - Changed types of read/writeRequestsCount to int.
          Hide
          stack added a comment -

          +1 on patch. Looks good Ted. Need to make sure though that we up the version on the Interface since the serialization of HSL has been changed by this patch (Unfortunately, HSL is not versioned so we can't have it migrate itself if it gets wrong version).

          Show
          stack added a comment - +1 on patch. Looks good Ted. Need to make sure though that we up the version on the Interface since the serialization of HSL has been changed by this patch (Unfortunately, HSL is not versioned so we can't have it migrate itself if it gets wrong version).
          Hide
          stack added a comment -

          Marking patch available but lets be careful about commit since we'll need a version bump on Interface before we can add this.

          Show
          stack added a comment - Marking patch available but lets be careful about commit since we'll need a version bump on Interface before we can add this.
          Hide
          stack added a comment -

          To be clear, for this patch to be committed, we will need to up our rpc version on all proxy interfaces. HServerLoad is buried inside HServerInfo at the moment. Its looking like pressure is building such that we will have to up our version numbers – hbase-1502 where we rejigger heartbeat includes deprecation of HSA and redo of HSI – but also Gary has patch to include cluster id in ClusterStatue.... So, the RPC version bump looks like it'll happen soon. We'll apply this patch then.

          Show
          stack added a comment - To be clear, for this patch to be committed, we will need to up our rpc version on all proxy interfaces. HServerLoad is buried inside HServerInfo at the moment. Its looking like pressure is building such that we will have to up our version numbers – hbase-1502 where we rejigger heartbeat includes deprecation of HSA and redo of HSI – but also Gary has patch to include cluster id in ClusterStatue.... So, the RPC version bump looks like it'll happen soon. We'll apply this patch then.
          Hide
          stack added a comment -

          Oh, we should add versioning to HSL and to CS and to HSI as part of version bump so in future they can self-migrate if they change.

          Show
          stack added a comment - Oh, we should add versioning to HSL and to CS and to HSI as part of version bump so in future they can self-migrate if they change.
          Hide
          Ted Yu added a comment -

          Realigned patch with current trunk.
          Also made HServerInfo and HServerLoad extend VersionedWritable.

          Show
          Ted Yu added a comment - Realigned patch with current trunk. Also made HServerInfo and HServerLoad extend VersionedWritable.
          Hide
          stack added a comment -

          Committed to TRUNK. My dementia on versions above should be ignored. We've already upped our rpc version across the board because of CoProcessors change. This change goes in under that same version change. Thanks for the patch Ted and you did us a service Versioning the HSI and HSL classes.

          Show
          stack added a comment - Committed to TRUNK. My dementia on versions above should be ignored. We've already upped our rpc version across the board because of CoProcessors change. This change goes in under that same version change. Thanks for the patch Ted and you did us a service Versioning the HSI and HSL classes.
          Hide
          stack added a comment -

          Ted, your patch is missing the new RegionLoad class. Thanks boss.

          Show
          stack added a comment - Ted, your patch is missing the new RegionLoad class. Thanks boss.
          Hide
          Ted Yu added a comment -

          Fix the error in HRegionServer.java

          Show
          Ted Yu added a comment - Fix the error in HRegionServer.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1824 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1824/)
          HBASE-3647 Distinguish read and write request count in region – reversed the patch because missing a file
          HBASE-3647 Distinguish read and write request count in region

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1824 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1824/ ) HBASE-3647 Distinguish read and write request count in region – reversed the patch because missing a file HBASE-3647 Distinguish read and write request count in region
          Hide
          stack added a comment -

          Applied Ted's v2. Compiles. Thanks again Ted.

          Show
          stack added a comment - Applied Ted's v2. Compiles. Thanks again Ted.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #1831 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1831/)

          Show
          Hudson added a comment - Integrated in HBase-TRUNK #1831 (See https://hudson.apache.org/hudson/job/HBase-TRUNK/1831/ )

            People

            • Assignee:
              Ted Yu
              Reporter:
              Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development