HBase
  1. HBase
  2. HBASE-4580

Some invalid zk nodes were created when a clean cluster restarts

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.92.0
    • Fix Version/s: 0.92.0
    • Component/s: master
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The below logs said that we created a invalid zk node when restarted a cluster.
      it mistakenly believed that the regions belong to a dead server.

      2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: Meta updated status = true
      2011-10-11 05:05:29,127 INFO org.apache.hadoop.hbase.master.HMaster: ROOT/Meta already up-to date with new HRI.
      2011-10-11 05:05:29,151 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for 771d63e9327383159553619a4f2dc74f with OFFLINE state
      2011-10-11 05:05:29,161 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for 3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
      2011-10-11 05:05:29,170 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for 4065350214452a9d5c55243c734bef08 with OFFLINE state
      2011-10-11 05:05:29,178 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for 4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
      2011-10-11 05:05:29,187 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
      2011-10-11 05:05:29,195 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for 5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
      2011-10-11 05:05:29,229 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for db5f641452a70b09b85a92970e4198c7 with OFFLINE state
      2011-10-11 05:05:29,237 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for a7b20a653919e7f41bfb2ed349af7d21 with OFFLINE state
      2011-10-11 05:05:29,253 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Creating (or updating) unassigned node for c9385619425f737eab1a6624d2e097a8 with OFFLINE state

      // we cleaned all zk nodes.
      2011-10-11 05:05:29,262 INFO org.apache.hadoop.hbase.master.AssignmentManager: Clean cluster startup. Assigning userregions
      2011-10-11 05:05:29,262 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Deleting any existing unassigned nodes
      2011-10-11 05:05:29,367 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 9 region(s) across 1 server(s), retainAssignment=true
      2011-10-11 05:05:29,369 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Timeout-on-RIT=9000
      2011-10-11 05:05:29,369 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning 9 region(s) to C3S3,54366,1318323920153
      2011-10-11 05:05:29,369 INFO org.apache.hadoop.hbase.master.AssignmentManager: Bulk assigning done
      2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for 771d63e9327383159553619a4f2dc74f with OFFLINE state
      2011-10-11 05:05:29,371 INFO org.apache.hadoop.hbase.master.HMaster: Master has completed initialization
      2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for 3cf860dd323fe6360f571aeafc129f95 with OFFLINE state
      2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for 4065350214452a9d5c55243c734bef08 with OFFLINE state
      2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for 4e81613f82a39fc6e5e89f96e7b3ccc4 with OFFLINE state
      2011-10-11 05:05:29,371 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for e21b9e1545a28953aba0098fda5c9cd9 with OFFLINE state
      2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for 5cd9f55eecd43d088bbd505f6795131f with OFFLINE state
      2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for db5f641452a70b09b85a92970e4198c7 with OFFLINE state
      2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for a7b20a653919e7f41bfb2ed349af7d21 with OFFLINE state
      2011-10-11 05:05:29,372 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:58198-0x132f23a9a380000 Async create of unassigned node for c9385619425f737eab1a6624d2e097a8 with OFFLINE state

      1. HBASE-4580_TrunkV3.patch
        8 kB
        gaojinchao
      2. HBASE-4580_TrunkV2.patch
        4 kB
        gaojinchao
      3. HBASE-4580_TrunkV1.patch
        4 kB
        gaojinchao

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            gaojinchao
            Reporter:
            gaojinchao
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Due:
              Created:
              Updated:
              Resolved:

              Development