Next Build

Regression

org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testOpenCloseRacing

Failing for the past 1 build (Since Failed#8362 )
Took 53 ms.

Error Message

Region should assigned on a new region server

Stacktrace

java.lang.AssertionError: Region should assigned on a new region server
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertFalse(Assert.java:64)
	at org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testOpenCloseRacing(TestAssignmentManagerOnCluster.java:631)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

Standard Output

2014-01-08 08:49:45,293 INFO  [pool-1-thread-1] hbase.HBaseTestingUtility(831): Starting up minicluster with 1 master(s) and 4 regionserver(s) and 4 datanode(s)
2014-01-08 08:49:45,332 INFO  [pool-1-thread-1] hbase.HBaseTestingUtility(388): Created new mini-cluster data directory: /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/target/test-data/be3c0b36-14c4-4e1e-aa1e-ce5fa5ea0950/dfscluster_68c9687d-afa1-4e26-9e5b-3dc12f6d5f64, deleteOnExit=true
2014-01-08 08:49:4
...[truncated 1317459 bytes]...
8118-1389170990546) service to localhost/127.0.0.1:49328
2014-01-08 08:50:23,002 WARN  [org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@1a4036f] blockmanagement.DecommissionManager$Monitor(78): Monitor interrupted: java.lang.InterruptedException: sleep interrupted
2014-01-08 08:50:23,071 INFO  [pool-1-thread-1] log.Slf4jLog(67): Stopped SelectChannelConnector@localhost:0
2014-01-08 08:50:23,225 INFO  [pool-1-thread-1] hbase.HBaseTestingUtility(952): Minicluster is down

Standard Error

2014-01-08 08:50:00,846 INFO  [pool-1-thread-1] hbase.ResourceChecker(147): before: master.TestAssignmentManagerOnCluster#testOpenCloseRacing Thread=489, OpenFileDescriptor=719, MaxFileDescriptor=60000, SystemLoadAverage=1701, ProcessCount=230, AvailableMemoryMB=2940, ConnectionCount=6
2014-01-08 08:50:00,851 INFO  [FifoRpcScheduler.handler1-thread-2] master.HMaster(1728): Client=jenkins//67.195.138.30 create 'testOpenCloseRacing', {NAME => 'FAMILY', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}
2014-01-08 08:50:00,869 DEBUG [FifoRpcScheduler.handler1-thread-2] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/testOpenCloseRacing/write-master:459550000000000
2014-01-08 08:50:00,892 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.CreateTableHandler(148): Create table testOpenCloseRacing
2014-01-08 08:50:00,945 INFO  [IPC Server handler 1 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:48118 is added to blk_1073741840_1016{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:56166|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:48118|RBW]]} size 0
2014-01-08 08:50:00,947 INFO  [IPC Server handler 6 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56166 is added to blk_1073741840_1016{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:56166|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:48118|RBW]]} size 0
2014-01-08 08:50:00,948 INFO  [IPC Server handler 4 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35049 is added to blk_1073741840_1016{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:56166|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:48118|RBW]]} size 0
2014-01-08 08:50:00,966 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] util.FSTableDescriptors(640): Wrote descriptor into: hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/.tabledesc/.tableinfo.0000000001
2014-01-08 08:50:00,979 INFO  [RegionOpenAndInitThread-testOpenCloseRacing-1] regionserver.HRegion(4155): creating HRegion testOpenCloseRacing HTD == 'testOpenCloseRacing', {NAME => 'FAMILY', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'} RootDir = hdfs://localhost:49328/user/jenkins/hbase/.tmp Table name == testOpenCloseRacing
2014-01-08 08:50:01,043 INFO  [IPC Server handler 2 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56166 is added to blk_1073741841_1017{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:51590|RBW], ReplicaUnderConstruction[127.0.0.1:48118|RBW], ReplicaUnderConstruction[127.0.0.1:56166|RBW]]} size 0
2014-01-08 08:50:01,047 INFO  [IPC Server handler 0 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:48118 is added to blk_1073741841_1017{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:51590|RBW], ReplicaUnderConstruction[127.0.0.1:48118|RBW], ReplicaUnderConstruction[127.0.0.1:56166|RBW]]} size 0
2014-01-08 08:50:01,050 INFO  [IPC Server handler 5 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51590 is added to blk_1073741841_1017 size 52
2014-01-08 08:50:01,053 DEBUG [RegionOpenAndInitThread-testOpenCloseRacing-1] regionserver.HRegion(567): Instantiated testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,053 DEBUG [RegionOpenAndInitThread-testOpenCloseRacing-1] regionserver.HRegion(1043): Closing testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.: disabling compactions & flushes
2014-01-08 08:50:01,054 DEBUG [RegionOpenAndInitThread-testOpenCloseRacing-1] regionserver.HRegion(1065): Updates disabled for region testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,054 INFO  [RegionOpenAndInitThread-testOpenCloseRacing-1] regionserver.HRegion(1123): Closed testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,100 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] catalog.MetaEditor(278): Added 1
2014-01-08 08:50:01,103 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] master.AssignmentManager(2503): Bulk assigning 1 region(s) across 4 server(s), round-robin=true
2014-01-08 08:50:01,115 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] master.AssignmentManager(1451): Assigning 1 region(s) to asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,115 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] zookeeper.ZKAssign(175): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Async create of unassigned node 6ab4725d0f6e44aa8a2845deaa5f49e8 with OFFLINE state
2014-01-08 08:50:01,122 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] master.GeneralBulkAssigner(177): Timeout-on-RIT=181000
2014-01-08 08:50:01,123 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:01,123 DEBUG [pool-1-thread-1-EventThread] master.OfflineCallback(69): rs={6ab4725d0f6e44aa8a2845deaa5f49e8 state=OFFLINE, ts=1389171001103, server=null}, server=asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,124 DEBUG [pool-1-thread-1-EventThread] master.OfflineCallback$ExistCallback(106): rs={6ab4725d0f6e44aa8a2845deaa5f49e8 state=OFFLINE, ts=1389171001103, server=null}, server=asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,124 INFO  [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] master.AssignmentManager(1502): asf002.sp2.ygridcore.net,36584,1389170993498 unassigned znodes=1 of total=1
2014-01-08 08:50:01,125 INFO  [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] master.RegionStates(316): Transitioned {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OFFLINE, ts=1389171001115, server=null} to {6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_OPEN, ts=1389171001125, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,125 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] master.ServerManager(765): New admin connection to asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,139 INFO  [Priority.RpcServer.handler=0,port=36584] regionserver.HRegionServer(3600): Open testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,150 DEBUG [RS_OPEN_REGION-asf002:36584-0] zookeeper.ZKAssign(832): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioning 6ab4725d0f6e44aa8a2845deaa5f49e8 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:01,152 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-GeneralBulkAssigner-0] master.AssignmentManager(1626): Bulk assigning done for asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,153 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] master.GeneralBulkAssigner(153): bulk assigning total 1 regions to 4 servers, took 31ms, with 1 regions still in transition
2014-01-08 08:50:01,153 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] master.AssignmentManager(2510): Bulk assigning done
2014-01-08 08:50:01,163 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/testOpenCloseRacing/write-master:459550000000000
2014-01-08 08:50:01,167 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,167 DEBUG [RS_OPEN_REGION-asf002:36584-0] zookeeper.ZKAssign(907): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioned node 6ab4725d0f6e44aa8a2845deaa5f49e8 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:01,168 DEBUG [RS_OPEN_REGION-asf002:36584-0] regionserver.HRegion(4341): Opening region: {ENCODED => 6ab4725d0f6e44aa8a2845deaa5f49e8, NAME => 'testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.', STARTKEY => '', ENDKEY => ''}
2014-01-08 08:50:01,169 INFO  [RS_OPEN_REGION-asf002:36584-0] coprocessor.CoprocessorHost(162): System coprocessor org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster$MyRegionObserver was loaded successfully with priority (536870911).
2014-01-08 08:50:01,169 DEBUG [RS_OPEN_REGION-asf002:36584-0] regionserver.MetricsRegionSourceImpl(64): Creating new MetricsRegionSourceImpl for table testOpenCloseRacing 6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,169 DEBUG [AM.ZK.Worker-pool2-t6] master.AssignmentManager(788): Handling RS_ZK_REGION_OPENING, server=asf002.sp2.ygridcore.net,36584,1389170993498, region=6ab4725d0f6e44aa8a2845deaa5f49e8, current_state={6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_OPEN, ts=1389171001125, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,169 INFO  [AM.ZK.Worker-pool2-t6] master.RegionStates(316): Transitioned {6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_OPEN, ts=1389171001125, server=asf002.sp2.ygridcore.net,36584,1389170993498} to {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPENING, ts=1389171001169, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,170 DEBUG [RS_OPEN_REGION-asf002:36584-0] regionserver.HRegion(567): Instantiated testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,186 INFO  [StoreOpener-6ab4725d0f6e44aa8a2845deaa5f49e8-1] compactions.CompactionConfiguration(88): size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; delete expired; major period 604800000, major jitter 0.500000
2014-01-08 08:50:01,192 DEBUG [RS_OPEN_REGION-asf002:36584-0] regionserver.HRegion(3012): Found 0 recovered edits file(s) under hdfs://localhost:49328/user/jenkins/hbase/data/default/testOpenCloseRacing/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,195 INFO  [RS_OPEN_REGION-asf002:36584-0] regionserver.HRegion(668): Onlined 6ab4725d0f6e44aa8a2845deaa5f49e8; next sequenceid=1
2014-01-08 08:50:01,195 DEBUG [RS_OPEN_REGION-asf002:36584-0] zookeeper.ZKAssign(644): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Attempting to retransition opening state of node 6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,199 INFO  [PostOpenDeployTasks:6ab4725d0f6e44aa8a2845deaa5f49e8] regionserver.HRegionServer(1754): Post open deploy tasks for region=testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,227 INFO  [PostOpenDeployTasks:6ab4725d0f6e44aa8a2845deaa5f49e8] catalog.MetaEditor(464): Updated row testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. with server=asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,227 INFO  [PostOpenDeployTasks:6ab4725d0f6e44aa8a2845deaa5f49e8] regionserver.HRegionServer(1779): Finished post open deploy task for testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:01,228 DEBUG [RS_OPEN_REGION-asf002:36584-0] zookeeper.ZKAssign(832): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioning 6ab4725d0f6e44aa8a2845deaa5f49e8 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED
2014-01-08 08:50:01,248 DEBUG [RS_OPEN_REGION-asf002:36584-0] zookeeper.ZKAssign(907): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioned node 6ab4725d0f6e44aa8a2845deaa5f49e8 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED
2014-01-08 08:50:01,248 DEBUG [RS_OPEN_REGION-asf002:36584-0] handler.OpenRegionHandler(379): Transitioned 6ab4725d0f6e44aa8a2845deaa5f49e8 to OPENED in zk on asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,249 DEBUG [RS_OPEN_REGION-asf002:36584-0] handler.OpenRegionHandler(179): Opened testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. on asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,250 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,260 DEBUG [AM.ZK.Worker-pool2-t8] master.AssignmentManager(788): Handling RS_ZK_REGION_OPENED, server=asf002.sp2.ygridcore.net,36584,1389170993498, region=6ab4725d0f6e44aa8a2845deaa5f49e8, current_state={6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPENING, ts=1389171001169, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,261 INFO  [AM.ZK.Worker-pool2-t8] master.RegionStates(316): Transitioned {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPENING, ts=1389171001169, server=asf002.sp2.ygridcore.net,36584,1389170993498} to {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPEN, ts=1389171001261, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,261 DEBUG [AM.ZK.Worker-pool2-t8] handler.OpenedRegionHandler(149): Handling OPENED of 6ab4725d0f6e44aa8a2845deaa5f49e8 from asf002.sp2.ygridcore.net,36584,1389170993498; deleting unassigned node
2014-01-08 08:50:01,269 DEBUG [AM.ZK.Worker-pool2-t8] zookeeper.ZKAssign(480): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Deleted unassigned node 6ab4725d0f6e44aa8a2845deaa5f49e8 in expected state RS_ZK_REGION_OPENED
2014-01-08 08:50:01,270 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:01,270 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:01,271 DEBUG [AM.ZK.Worker-pool2-t9] master.AssignmentManager$4(1173): Znode testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. deleted, state: {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPEN, ts=1389171001261, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:01,271 INFO  [AM.ZK.Worker-pool2-t9] master.RegionStates(377): Onlined 6ab4725d0f6e44aa8a2845deaa5f49e8 on asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:01,543 INFO  [Thread-415] zookeeper.RecoverableZooKeeper(120): Process identifier=catalogtracker-on-hconnection-0x11c1463 connecting to ZooKeeper ensemble=localhost:55976
2014-01-08 08:50:01,543 DEBUG [Thread-415] catalog.CatalogTracker(194): Starting catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1158bbe
2014-01-08 08:50:01,547 DEBUG [Thread-415] zookeeper.ZKUtil(428): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
2014-01-08 08:50:01,547 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null
2014-01-08 08:50:01,549 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(387): catalogtracker-on-hconnection-0x11c1463-0x143710b250b0014 connected
2014-01-08 08:50:01,564 DEBUG [Thread-415] catalog.CatalogTracker(210): Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1158bbe
2014-01-08 08:50:01,583 DEBUG [Thread-415] catalog.MetaEditor(238): Added testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:01,584 WARN  [Thread-415] master.AssignmentManager(1751): Assigning a region not in region states: {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'}
2014-01-08 08:50:01,584 DEBUG [Thread-415] master.AssignmentManager(2181): No previous transition plan found (or ignoring an existing plan) for testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.; generated random plan=hri=testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263., src=, dest=asf002.sp2.ygridcore.net,59479,1389170993670; 4 (online=4, available=4) available servers, forceNewPlan=false
2014-01-08 08:50:01,584 DEBUG [Thread-415] zookeeper.ZKAssign(206): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Creating (or updating) unassigned node c18ad6dfb0055258336e96a299f57263 with OFFLINE state
2014-01-08 08:50:01,591 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:01,593 INFO  [Thread-415] master.AssignmentManager(1910): Assigning testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. to asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:01,593 INFO  [Thread-415] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=OFFLINE, ts=1389171001584, server=null} to {c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171001593, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,594 INFO  [Priority.RpcServer.handler=6,port=59479] regionserver.HRegionServer(3600): Open testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:01,608 DEBUG [RS_OPEN_REGION-asf002:59479-2] zookeeper.ZKAssign(832): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioning c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:01,617 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:01,617 DEBUG [RS_OPEN_REGION-asf002:59479-2] zookeeper.ZKAssign(907): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioned node c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:01,618 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(4341): Opening region: {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'}
2014-01-08 08:50:01,619 INFO  [RS_OPEN_REGION-asf002:59479-2] coprocessor.CoprocessorHost(162): System coprocessor org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster$MyRegionObserver was loaded successfully with priority (536870911).
2014-01-08 08:50:01,619 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.MetricsRegionSourceImpl(64): Creating new MetricsRegionSourceImpl for table testOpenCloseRacing c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:01,619 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(567): Instantiated testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:01,622 WARN  [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegionFileSystem(742): .regioninfo file not found for region: c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:01,623 DEBUG [AM.ZK.Worker-pool2-t13] master.AssignmentManager(788): Handling RS_ZK_REGION_OPENING, server=asf002.sp2.ygridcore.net,59479,1389170993670, region=c18ad6dfb0055258336e96a299f57263, current_state={c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171001593, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,624 INFO  [AM.ZK.Worker-pool2-t13] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171001593, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171001624, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,670 INFO  [IPC Server handler 9 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56166 is added to blk_1073741842_1018{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:51590|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:56166|RBW]]} size 0
2014-01-08 08:50:01,672 INFO  [IPC Server handler 3 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51590 is added to blk_1073741842_1018{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:51590|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:56166|RBW]]} size 0
2014-01-08 08:50:01,673 INFO  [IPC Server handler 1 on 49328] blockmanagement.BlockManager(2237): BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35049 is added to blk_1073741842_1018{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[127.0.0.1:51590|RBW], ReplicaUnderConstruction[127.0.0.1:35049|RBW], ReplicaUnderConstruction[127.0.0.1:56166|RBW]]} size 0
2014-01-08 08:50:01,712 INFO  [StoreOpener-c18ad6dfb0055258336e96a299f57263-1] compactions.CompactionConfiguration(88): size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; delete expired; major period 604800000, major jitter 0.500000
2014-01-08 08:50:01,721 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(3012): Found 0 recovered edits file(s) under hdfs://localhost:49328/user/jenkins/hbase/data/default/testOpenCloseRacing/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:01,725 INFO  [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(668): Onlined c18ad6dfb0055258336e96a299f57263; next sequenceid=1
2014-01-08 08:50:01,904 DEBUG [Thread-415] master.AssignmentManager(2229): Starting unassign of testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. (offlining), current state: {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171001624, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,905 DEBUG [Thread-415] master.AssignmentManager(2314): Attempting to unassign testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. but it is already in transition (OPENING, force=false)
2014-01-08 08:50:01,905 DEBUG [Thread-415] master.AssignmentManager(1757): Force region state offline {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171001624, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,906 INFO  [Priority.RpcServer.handler=7,port=59479] regionserver.HRegionServer(3722): Close c18ad6dfb0055258336e96a299f57263, via zk=no, znode version=-1, on null
2014-01-08 08:50:01,906 INFO  [Priority.RpcServer.handler=7,port=59479] regionserver.HRegionServer(2539): Received CLOSE for the region:c18ad6dfb0055258336e96a299f57263 , which we are already trying to OPEN. Cancelling OPENING.
2014-01-08 08:50:01,906 INFO  [Priority.RpcServer.handler=7,port=59479] regionserver.HRegionServer(2551): The opening previously in progress has been cancelled by a CLOSE request.
2014-01-08 08:50:01,908 DEBUG [Thread-415] master.AssignmentManager(1694): Offline testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263., it's not any more on asf002.sp2.ygridcore.net,59479,1389170993670
org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException: The region c18ad6dfb0055258336e96a299f57263 was opening but not yet served. Opening is cancelled.
	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2553)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:3725)
	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:19797)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
	at java.lang.Thread.run(Thread.java:662)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:280)
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.closeRegion(ProtobufUtil.java:1594)
	at org.apache.hadoop.hbase.master.ServerManager.sendRegionClose(ServerManager.java:693)
	at org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1672)
	at org.apache.hadoop.hbase.master.AssignmentManager.forceRegionStateToOffline(AssignmentManager.java:1773)
	at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1423)
	at org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster.testOpenCloseRacing(TestAssignmentManagerOnCluster.java:622)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.NotServingRegionException): org.apache.hadoop.hbase.NotServingRegionException: The region c18ad6dfb0055258336e96a299f57263 was opening but not yet served. Opening is cancelled.
	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2553)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:3725)
	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:19797)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
	at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
	at java.lang.Thread.run(Thread.java:662)

	at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1450)
	at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
	at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.closeRegion(AdminProtos.java:20607)
	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.closeRegion(ProtobufUtil.java:1591)
	... 14 more
2014-01-08 08:50:01,911 INFO  [Thread-415] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171001624, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=OFFLINE, ts=1389171001911, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:01,919 DEBUG [Thread-415] master.AssignmentManager(2181): No previous transition plan found (or ignoring an existing plan) for testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.; generated random plan=hri=testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263., src=, dest=asf002.sp2.ygridcore.net,59479,1389170993670; 4 (online=4, available=4) available servers, forceNewPlan=true
2014-01-08 08:50:01,919 DEBUG [Thread-415] zookeeper.ZKAssign(206): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Creating (or updating) unassigned node c18ad6dfb0055258336e96a299f57263 with OFFLINE state
2014-01-08 08:50:02,726 WARN  [RS_OPEN_REGION-asf002:59479-2] handler.OpenRegionHandler(547): Open region aborted since it isn't opening any more
2014-01-08 08:50:02,726 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(1043): Closing testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.: disabling compactions & flushes
2014-01-08 08:50:02,726 DEBUG [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(1065): Updates disabled for region testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:02,727 INFO  [StoreCloserThread-testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.-1] regionserver.HStore(740): Closed FAMILY
2014-01-08 08:50:02,728 INFO  [RS_OPEN_REGION-asf002:59479-2] regionserver.HRegion(1123): Closed testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:02,728 INFO  [RS_OPEN_REGION-asf002:59479-2] handler.OpenRegionHandler(399): Opening of region {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'} failed, transitioning from OPENING to FAILED_OPEN in ZK, expecting version 1
2014-01-08 08:50:02,728 DEBUG [RS_OPEN_REGION-asf002:59479-2] zookeeper.ZKAssign(832): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioning c18ad6dfb0055258336e96a299f57263 from RS_ZK_REGION_OPENING to RS_ZK_REGION_FAILED_OPEN
2014-01-08 08:50:02,783 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:02,784 WARN  [RS_OPEN_REGION-asf002:59479-2] zookeeper.ZKAssign(849): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Attempt to transition the unassigned node for c18ad6dfb0055258336e96a299f57263 from RS_ZK_REGION_OPENING to RS_ZK_REGION_FAILED_OPEN failed, the node existed but was version 2 not the expected version 1
2014-01-08 08:50:02,784 WARN  [RS_OPEN_REGION-asf002:59479-2] handler.OpenRegionHandler(407): Unable to mark region {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'} as FAILED_OPEN. It's likely that the master already timed out this open attempt, and thus another RS already has the region.
2014-01-08 08:50:02,786 DEBUG [AM.ZK.Worker-pool2-t12] master.AssignmentManager(788): Handling M_ZK_REGION_OFFLINE, server=asf002.sp2.ygridcore.net,59479,1389170993670, region=c18ad6dfb0055258336e96a299f57263, current_state={c18ad6dfb0055258336e96a299f57263 state=OFFLINE, ts=1389171001919, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:02,786 INFO  [Thread-415] master.AssignmentManager(1910): Assigning testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. to asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:02,787 INFO  [Thread-415] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=OFFLINE, ts=1389171001919, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171002787, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:02,788 INFO  [Priority.RpcServer.handler=8,port=59479] regionserver.HRegionServer(3600): Open testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:02,792 DEBUG [RS_OPEN_REGION-asf002:59479-0] zookeeper.ZKAssign(832): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioning c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:02,794 INFO  [Thread-415] master.AssignmentManager(2837): Waiting for c18ad6dfb0055258336e96a299f57263 to leave regions-in-transition, timeOut=-1 ms.
2014-01-08 08:50:03,335 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,336 DEBUG [RS_OPEN_REGION-asf002:59479-0] zookeeper.ZKAssign(907): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioned node c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-01-08 08:50:03,337 DEBUG [RS_OPEN_REGION-asf002:59479-0] regionserver.HRegion(4341): Opening region: {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'}
2014-01-08 08:50:03,337 INFO  [RS_OPEN_REGION-asf002:59479-0] coprocessor.CoprocessorHost(162): System coprocessor org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster$MyRegionObserver was loaded successfully with priority (536870911).
2014-01-08 08:50:03,337 DEBUG [RS_OPEN_REGION-asf002:59479-0] regionserver.MetricsRegionSourceImpl(64): Creating new MetricsRegionSourceImpl for table testOpenCloseRacing c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,338 DEBUG [RS_OPEN_REGION-asf002:59479-0] regionserver.HRegion(567): Instantiated testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,338 DEBUG [AM.ZK.Worker-pool2-t14] master.AssignmentManager(788): Handling RS_ZK_REGION_OPENING, server=asf002.sp2.ygridcore.net,59479,1389170993670, region=c18ad6dfb0055258336e96a299f57263, current_state={c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171002787, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,338 INFO  [AM.ZK.Worker-pool2-t14] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=PENDING_OPEN, ts=1389171002787, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171003338, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,354 INFO  [StoreOpener-c18ad6dfb0055258336e96a299f57263-1] compactions.CompactionConfiguration(88): size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; delete expired; major period 604800000, major jitter 0.500000
2014-01-08 08:50:03,359 DEBUG [RS_OPEN_REGION-asf002:59479-0] regionserver.HRegion(3012): Found 0 recovered edits file(s) under hdfs://localhost:49328/user/jenkins/hbase/data/default/testOpenCloseRacing/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,362 INFO  [RS_OPEN_REGION-asf002:59479-0] regionserver.HRegion(668): Onlined c18ad6dfb0055258336e96a299f57263; next sequenceid=1
2014-01-08 08:50:03,362 DEBUG [RS_OPEN_REGION-asf002:59479-0] zookeeper.ZKAssign(644): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Attempting to retransition opening state of node c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,366 INFO  [PostOpenDeployTasks:c18ad6dfb0055258336e96a299f57263] regionserver.HRegionServer(1754): Post open deploy tasks for region=testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,385 INFO  [PostOpenDeployTasks:c18ad6dfb0055258336e96a299f57263] catalog.MetaEditor(464): Updated row testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. with server=asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,385 INFO  [PostOpenDeployTasks:c18ad6dfb0055258336e96a299f57263] regionserver.HRegionServer(1779): Finished post open deploy task for testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,385 DEBUG [RS_OPEN_REGION-asf002:59479-0] zookeeper.ZKAssign(832): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioning c18ad6dfb0055258336e96a299f57263 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED
2014-01-08 08:50:03,391 DEBUG [RS_OPEN_REGION-asf002:59479-0] zookeeper.ZKAssign(907): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioned node c18ad6dfb0055258336e96a299f57263 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED
2014-01-08 08:50:03,391 DEBUG [RS_OPEN_REGION-asf002:59479-0] handler.OpenRegionHandler(379): Transitioned c18ad6dfb0055258336e96a299f57263 to OPENED in zk on asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,391 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,391 DEBUG [RS_OPEN_REGION-asf002:59479-0] handler.OpenRegionHandler(179): Opened testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. on asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,393 DEBUG [AM.ZK.Worker-pool2-t15] master.AssignmentManager(788): Handling RS_ZK_REGION_OPENED, server=asf002.sp2.ygridcore.net,59479,1389170993670, region=c18ad6dfb0055258336e96a299f57263, current_state={c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171003338, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,393 INFO  [AM.ZK.Worker-pool2-t15] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=OPENING, ts=1389171003338, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=OPEN, ts=1389171003393, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,394 DEBUG [AM.ZK.Worker-pool2-t15] handler.OpenedRegionHandler(149): Handling OPENED of c18ad6dfb0055258336e96a299f57263 from asf002.sp2.ygridcore.net,59479,1389170993670; deleting unassigned node
2014-01-08 08:50:03,404 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,405 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:03,405 DEBUG [AM.ZK.Worker-pool2-t15] zookeeper.ZKAssign(480): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Deleted unassigned node c18ad6dfb0055258336e96a299f57263 in expected state RS_ZK_REGION_OPENED
2014-01-08 08:50:03,405 DEBUG [AM.ZK.Worker-pool2-t16] master.AssignmentManager$4(1173): Znode testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. deleted, state: {c18ad6dfb0055258336e96a299f57263 state=OPEN, ts=1389171003393, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,406 INFO  [AM.ZK.Worker-pool2-t16] master.RegionStates(377): Onlined c18ad6dfb0055258336e96a299f57263 on asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,417 INFO  [Thread-415] client.HBaseAdmin$5(909): Started disable of testOpenCloseRacing
2014-01-08 08:50:03,419 INFO  [FifoRpcScheduler.handler1-thread-5] master.HMaster(1960): Client=jenkins//67.195.138.30 disable testOpenCloseRacing
2014-01-08 08:50:03,425 DEBUG [FifoRpcScheduler.handler1-thread-5] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/testOpenCloseRacing/write-master:459550000000001
2014-01-08 08:50:03,437 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DisableTableHandler(130): Attempting to disable table testOpenCloseRacing
2014-01-08 08:50:03,443 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DisableTableHandler(174): Offlining 2 regions.
2014-01-08 08:50:03,451 INFO  [Thread-415] zookeeper.RecoverableZooKeeper(120): Process identifier=catalogtracker-on-hconnection-0x11c1463 connecting to ZooKeeper ensemble=localhost:55976
2014-01-08 08:50:03,451 DEBUG [Thread-415] catalog.CatalogTracker(194): Starting catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1fa350b
2014-01-08 08:50:03,467 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null
2014-01-08 08:50:03,468 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0] master.AssignmentManager(2229): Starting unassign of testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. (offlining), current state: {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPEN, ts=1389171001271, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:03,468 DEBUG [Thread-415] zookeeper.ZKUtil(428): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
2014-01-08 08:50:03,468 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1] master.AssignmentManager(2229): Starting unassign of testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. (offlining), current state: {c18ad6dfb0055258336e96a299f57263 state=OPEN, ts=1389171003406, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,468 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1] zookeeper.ZKAssign(527): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Creating unassigned node c18ad6dfb0055258336e96a299f57263 in a CLOSING state
2014-01-08 08:50:03,468 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0] zookeeper.ZKAssign(527): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Creating unassigned node 6ab4725d0f6e44aa8a2845deaa5f49e8 in a CLOSING state
2014-01-08 08:50:03,471 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:03,479 INFO  [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=OPEN, ts=1389171003406, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=PENDING_CLOSE, ts=1389171003479, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,480 INFO  [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0] master.RegionStates(316): Transitioned {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OPEN, ts=1389171001271, server=asf002.sp2.ygridcore.net,36584,1389170993498} to {6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_CLOSE, ts=1389171003480, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:03,481 INFO  [Priority.RpcServer.handler=5,port=59479] regionserver.HRegionServer(3722): Close c18ad6dfb0055258336e96a299f57263, via zk=yes, znode version=0, on null
2014-01-08 08:50:03,482 INFO  [Priority.RpcServer.handler=1,port=36584] regionserver.HRegionServer(3722): Close 6ab4725d0f6e44aa8a2845deaa5f49e8, via zk=yes, znode version=0, on null
2014-01-08 08:50:03,483 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(387): catalogtracker-on-hconnection-0x11c1463-0x143710b250b0015 connected
2014-01-08 08:50:03,488 DEBUG [RS_CLOSE_REGION-asf002:59479-1] handler.CloseRegionHandler(125): Processing close of testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,489 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-0] master.AssignmentManager(1674): Sent CLOSE to asf002.sp2.ygridcore.net,36584,1389170993498 for region testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,489 DEBUG [RS_CLOSE_REGION-asf002:36584-0] handler.CloseRegionHandler(125): Processing close of testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,490 DEBUG [asf002.sp2.ygridcore.net,45955,1389170993145-org.apache.hadoop.hbase.master.handler.DisableTableHandler$BulkDisabler-1] master.AssignmentManager(1674): Sent CLOSE to asf002.sp2.ygridcore.net,59479,1389170993670 for region testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,494 DEBUG [RS_CLOSE_REGION-asf002:59479-1] regionserver.HRegion(1043): Closing testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.: disabling compactions & flushes
2014-01-08 08:50:03,494 DEBUG [RS_CLOSE_REGION-asf002:59479-1] regionserver.HRegion(1065): Updates disabled for region testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,499 DEBUG [RS_CLOSE_REGION-asf002:36584-0] regionserver.HRegion(1043): Closing testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.: disabling compactions & flushes
2014-01-08 08:50:03,499 DEBUG [RS_CLOSE_REGION-asf002:36584-0] regionserver.HRegion(1065): Updates disabled for region testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,502 INFO  [StoreCloserThread-testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.-1] regionserver.HStore(740): Closed FAMILY
2014-01-08 08:50:03,502 INFO  [RS_CLOSE_REGION-asf002:59479-1] regionserver.HRegion(1123): Closed testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,502 DEBUG [RS_CLOSE_REGION-asf002:59479-1] zookeeper.ZKAssign(832): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioning c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_CLOSING to RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,502 INFO  [StoreCloserThread-testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.-1] regionserver.HStore(740): Closed FAMILY
2014-01-08 08:50:03,503 DEBUG [Thread-415] catalog.CatalogTracker(210): Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1fa350b
2014-01-08 08:50:03,505 INFO  [RS_CLOSE_REGION-asf002:36584-0] regionserver.HRegion(1123): Closed testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,505 DEBUG [RS_CLOSE_REGION-asf002:36584-0] zookeeper.ZKAssign(832): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioning 6ab4725d0f6e44aa8a2845deaa5f49e8 from M_ZK_REGION_CLOSING to RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,518 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done; 300000 ms remaining; [{ENCODED => 6ab4725d0f6e44aa8a2845deaa5f49e8, NAME => 'testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'}]
2014-01-08 08:50:03,534 DEBUG [RS_CLOSE_REGION-asf002:59479-1] zookeeper.ZKAssign(907): regionserver:59479-0x143710b250b0004, quorum=localhost:55976, baseZNode=/hbase Transitioned node c18ad6dfb0055258336e96a299f57263 from M_ZK_REGION_CLOSING to RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,534 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,535 DEBUG [Thread-415] client.HBaseAdmin(948): Sleeping= 100ms, waiting for all regions to be disabled in testOpenCloseRacing
2014-01-08 08:50:03,535 DEBUG [RS_CLOSE_REGION-asf002:59479-1] handler.CloseRegionHandler(168): Set closed state in zk for testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. on asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,536 DEBUG [RS_CLOSE_REGION-asf002:59479-1] handler.CloseRegionHandler(176): Closed testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,539 DEBUG [RS_CLOSE_REGION-asf002:36584-0] zookeeper.ZKAssign(907): regionserver:36584-0x143710b250b0001, quorum=localhost:55976, baseZNode=/hbase Transitioned node 6ab4725d0f6e44aa8a2845deaa5f49e8 from M_ZK_REGION_CLOSING to RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,539 DEBUG [RS_CLOSE_REGION-asf002:36584-0] handler.CloseRegionHandler(168): Set closed state in zk for testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. on asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:03,539 DEBUG [RS_CLOSE_REGION-asf002:36584-0] handler.CloseRegionHandler(176): Closed testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,539 DEBUG [AM.ZK.Worker-pool2-t19] master.AssignmentManager(788): Handling RS_ZK_REGION_CLOSED, server=asf002.sp2.ygridcore.net,59479,1389170993670, region=c18ad6dfb0055258336e96a299f57263, current_state={c18ad6dfb0055258336e96a299f57263 state=PENDING_CLOSE, ts=1389171003479, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,539 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:03,539 DEBUG [AM.ZK.Worker-pool2-t19] handler.ClosedRegionHandler(92): Handling CLOSED event for c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,540 DEBUG [AM.ZK.Worker-pool2-t19] master.AssignmentManager(1374): Table being disabled so deleting ZK node and removing from regions in transition, skipping assignment of region testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.
2014-01-08 08:50:03,542 DEBUG [AM.ZK.Worker-pool2-t3] master.AssignmentManager(788): Handling RS_ZK_REGION_CLOSED, server=asf002.sp2.ygridcore.net,36584,1389170993498, region=6ab4725d0f6e44aa8a2845deaa5f49e8, current_state={6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_CLOSE, ts=1389171003480, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:03,542 DEBUG [AM.ZK.Worker-pool2-t3] handler.ClosedRegionHandler(92): Handling CLOSED event for 6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:03,542 DEBUG [AM.ZK.Worker-pool2-t3] master.AssignmentManager(1374): Table being disabled so deleting ZK node and removing from regions in transition, skipping assignment of region testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.
2014-01-08 08:50:03,553 DEBUG [AM.ZK.Worker-pool2-t19] zookeeper.ZKAssign(480): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Deleted unassigned node c18ad6dfb0055258336e96a299f57263 in expected state RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,553 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,553 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/region-in-transition
2014-01-08 08:50:03,553 INFO  [AM.ZK.Worker-pool2-t19] master.RegionStates(316): Transitioned {c18ad6dfb0055258336e96a299f57263 state=PENDING_CLOSE, ts=1389171003479, server=asf002.sp2.ygridcore.net,59479,1389170993670} to {c18ad6dfb0055258336e96a299f57263 state=OFFLINE, ts=1389171003553, server=asf002.sp2.ygridcore.net,59479,1389170993670}
2014-01-08 08:50:03,554 INFO  [AM.ZK.Worker-pool2-t19] master.RegionStates(461): Offlined c18ad6dfb0055258336e96a299f57263 from asf002.sp2.ygridcore.net,59479,1389170993670
2014-01-08 08:50:03,561 DEBUG [pool-1-thread-1-EventThread] zookeeper.ZooKeeperWatcher(310): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/region-in-transition/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:03,562 DEBUG [AM.ZK.Worker-pool2-t3] zookeeper.ZKAssign(480): master:45955-0x143710b250b0000, quorum=localhost:55976, baseZNode=/hbase Deleted unassigned node 6ab4725d0f6e44aa8a2845deaa5f49e8 in expected state RS_ZK_REGION_CLOSED
2014-01-08 08:50:03,562 INFO  [AM.ZK.Worker-pool2-t3] master.RegionStates(316): Transitioned {6ab4725d0f6e44aa8a2845deaa5f49e8 state=PENDING_CLOSE, ts=1389171003480, server=asf002.sp2.ygridcore.net,36584,1389170993498} to {6ab4725d0f6e44aa8a2845deaa5f49e8 state=OFFLINE, ts=1389171003562, server=asf002.sp2.ygridcore.net,36584,1389170993498}
2014-01-08 08:50:03,562 INFO  [AM.ZK.Worker-pool2-t3] master.RegionStates(461): Offlined 6ab4725d0f6e44aa8a2845deaa5f49e8 from asf002.sp2.ygridcore.net,36584,1389170993498
2014-01-08 08:50:03,568 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DisableTableHandler$BulkDisabler(230): Disable waiting until done; 299950 ms remaining; []
2014-01-08 08:50:03,577 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DisableTableHandler(190): Disabled table is done=true
2014-01-08 08:50:03,584 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/testOpenCloseRacing/write-master:459550000000001
2014-01-08 08:50:03,642 INFO  [Thread-415] zookeeper.RecoverableZooKeeper(120): Process identifier=catalogtracker-on-hconnection-0x11c1463 connecting to ZooKeeper ensemble=localhost:55976
2014-01-08 08:50:03,642 DEBUG [Thread-415] catalog.CatalogTracker(194): Starting catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1f42fbb
2014-01-08 08:50:03,825 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(310): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null
2014-01-08 08:50:03,825 DEBUG [Thread-415] zookeeper.ZKUtil(428): catalogtracker-on-hconnection-0x11c1463, quorum=localhost:55976, baseZNode=/hbase Set watcher on existing znode=/hbase/meta-region-server
2014-01-08 08:50:03,826 DEBUG [Thread-415-EventThread] zookeeper.ZooKeeperWatcher(387): catalogtracker-on-hconnection-0x11c1463-0x143710b250b0016 connected
2014-01-08 08:50:03,842 DEBUG [Thread-415] catalog.CatalogTracker(210): Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@1f42fbb
2014-01-08 08:50:03,847 INFO  [Thread-415] client.HBaseAdmin(964): Disabled testOpenCloseRacing
2014-01-08 08:50:03,854 INFO  [FifoRpcScheduler.handler1-thread-2] master.HMaster(1797): Client=jenkins//67.195.138.30 delete testOpenCloseRacing
2014-01-08 08:50:03,860 DEBUG [FifoRpcScheduler.handler1-thread-2] lock.ZKInterProcessLockBase(226): Acquired a lock for /hbase/table-lock/testOpenCloseRacing/write-master:459550000000002
2014-01-08 08:50:03,891 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.TableEventHandler(123): Handling table operation C_M_DELETE_TABLE on table testOpenCloseRacing
2014-01-08 08:50:03,901 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(93): Deleting regions from META
2014-01-08 08:50:03,912 INFO  [MASTER_TABLE_OPERATIONS-asf002:45955-0] catalog.MetaEditor(495): Deleted [{ENCODED => 6ab4725d0f6e44aa8a2845deaa5f49e8, NAME => 'testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c18ad6dfb0055258336e96a299f57263, NAME => 'testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263.', STARTKEY => 'A', ENDKEY => 'Z'}]
2014-01-08 08:50:03,926 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(104): Archiving region testOpenCloseRacing,,1389171000851.6ab4725d0f6e44aa8a2845deaa5f49e8. from FS
2014-01-08 08:50:03,926 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(93): ARCHIVING hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:03,933 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(134): Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/6ab4725d0f6e44aa8a2845deaa5f49e8/FAMILY]
2014-01-08 08:50:03,954 INFO  [IPC Server handler 0 on 49328] blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: blk_1073741841_1017 127.0.0.1:48118 127.0.0.1:56166 127.0.0.1:51590 
2014-01-08 08:50:03,955 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(453): Deleted all region files in: hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/6ab4725d0f6e44aa8a2845deaa5f49e8
2014-01-08 08:50:03,955 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(104): Archiving region testOpenCloseRacing,A,1389171001573.c18ad6dfb0055258336e96a299f57263. from FS
2014-01-08 08:50:03,955 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(93): ARCHIVING hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,961 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(134): Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/c18ad6dfb0055258336e96a299f57263/FAMILY]
2014-01-08 08:50:03,976 INFO  [IPC Server handler 2 on 49328] blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: blk_1073741842_1018 127.0.0.1:35049 127.0.0.1:51590 127.0.0.1:56166 
2014-01-08 08:50:03,977 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] backup.HFileArchiver(453): Deleted all region files in: hdfs://localhost:49328/user/jenkins/hbase/.tmp/data/default/testOpenCloseRacing/c18ad6dfb0055258336e96a299f57263
2014-01-08 08:50:03,981 INFO  [IPC Server handler 0 on 49328] blockmanagement.BlockManager(1009): BLOCK* addToInvalidates: blk_1073741840_1016 127.0.0.1:35049 127.0.0.1:56166 127.0.0.1:48118 
2014-01-08 08:50:03,982 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(114): Table 'testOpenCloseRacing' archived!
2014-01-08 08:50:03,982 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(117): Removing 'testOpenCloseRacing' descriptor.
2014-01-08 08:50:03,983 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] handler.DeleteTableHandler(121): Marking 'testOpenCloseRacing' as deleted.
2014-01-08 08:50:03,991 DEBUG [MASTER_TABLE_OPERATIONS-asf002:45955-0] lock.ZKInterProcessLockBase(328): Released /hbase/table-lock/testOpenCloseRacing/write-master:459550000000002
2014-01-08 08:50:04,001 INFO  [Thread-415] client.HBaseAdmin(696): Deleted testOpenCloseRacing
2014-01-08 08:50:04,043 INFO  [pool-1-thread-1] hbase.ResourceChecker(171): after: master.TestAssignmentManagerOnCluster#testOpenCloseRacing Thread=500 (was 489)
Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/target/test-data/be3c0b36-14c4-4e1e-aa1e-ce5fa5ea0950/dfscluster_68c9687d-afa1-4e26-9e5b-3dc12f6d5f64/dfs/data/data7/current
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
	java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: RS_OPEN_REGION-asf002:59479-2
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/target/test-data/be3c0b36-14c4-4e1e-aa1e-ce5fa5ea0950/dfscluster_68c9687d-afa1-4e26-9e5b-3dc12f6d5f64/dfs/data/data8/current
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
	java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: IPC Client (27940994) connection to asf002.sp2.ygridcore.net/67.195.138.30:39238 from jenkins.hfs.0
	java.lang.Object.wait(Native Method)
	org.apache.hadoop.hbase.ipc.RpcClient$Connection.waitForWork(RpcClient.java:674)
	org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:720)

Potentially hanging thread: RS_CLOSE_REGION-asf002:59479-1
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/target/test-data/be3c0b36-14c4-4e1e-aa1e-ce5fa5ea0950/dfscluster_68c9687d-afa1-4e26-9e5b-3dc12f6d5f64/dfs/data/data2/current
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
	java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/target/test-data/be3c0b36-14c4-4e1e-aa1e-ce5fa5ea0950/dfscluster_68c9687d-afa1-4e26-9e5b-3dc12f6d5f64/dfs/data/data1/current
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
	java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: htable-pool33-t1
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
	java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
	java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
	java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1088788711_8 at /127.0.0.1:60001 [Waiting for operation #2]
	sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
	sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
	sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
	sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
	sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
	org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
	org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
	org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
	org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
	java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	java.io.BufferedInputStream.read(BufferedInputStream.java:237)
	java.io.DataInputStream.readShort(DataInputStream.java:295)
	org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:52)
	org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:198)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: RS_CLOSE_REGION-asf002:36584-0
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)

Potentially hanging thread: IPC Client (27940994) connection to asf002.sp2.ygridcore.net/67.195.138.30:36584 from jenkins
	java.lang.Object.wait(Native Method)
	org.apache.hadoop.hbase.ipc.RpcClient$Connection.waitForWork(RpcClient.java:674)
	org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:720)

Potentially hanging thread: RS_OPEN_REGION-asf002:36584-0
	sun.misc.Unsafe.park(Native Method)
	java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
	java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
	java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
	java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
	java.lang.Thread.run(Thread.java:662)
 - Thread LEAK? -, OpenFileDescriptor=729 (was 719) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1701 (was 1701), ProcessCount=230 (was 230), AvailableMemoryMB=2970 (was 2940) - AvailableMemoryMB LEAK? -, ConnectionCount=6 (was 6)