Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23958

Balancer keeps balancing indefinitely

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.2
    • None
    • None

    Description

      Before raising this issue - am not sure if this got fixed directly or indirectly in other latest versions of hbase.
      The steps are
      1) Create a cluster and create some tables. (assume we have RS 1,2,3, 4 and 5)
      2) After the table creation and some ops done, the cluster was restarted. Due to this some regions are in RIT. the RIT in progress was to be assigned to RS 3.
      3) After the cluster comes back RS 3 and 4 are stopped. (RS 3 will have newer timestamp)
      4) Now the master that comes up sees there are some RIT in place and tries to load the entries to process the procedures again. As part of this the RegionStateStore is populated with the old RS 3 hostname. (older timestamp). This adds to the ServerStateNode creating a RS 3 with old timestamp as one server.
      5) Now after the master restarts and all regions assigned, the balancer infinitely tries to balance the region to the RS 3 (old timestamp server) thinking it is part of the cluster.
      6)the other problem is the MoveProcedure has the target as RS 3 (with old timestamp) but the AM realizes that it is a down server and move it to the one of the active server. But this is not recorded anywhere.
      I will continue to check the latest code if this case is valid.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ram_krish ramkrishna.s.vasudevan
            ram_krish ramkrishna.s.vasudevan

            Dates

              Created:
              Updated:

              Slack

                Issue deployment