2007-08-30 08:20:19,550 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:19,561 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo (no reconstruction log) 2007-08-30 08:20:19,607 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo is 3224347 2007-08-30 08:20:19,631 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 is 3224348 2007-08-30 08:20:19,631 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 available 2007-08-30 08:20:19,631 DEBUG org.apache.hadoop.hbase.HLog: changing sequence number from 3223587 to 3224348 2007-08-30 08:20:19,631 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,QhG8ZBSycYccekKNqw-UKF==,1369474941778185308, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:19,633 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,QhG8ZBSycYccekKNqw-UKF==,1369474941778185308/repo (no reconstruction log) 2007-08-30 08:20:19,647 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,QhG8ZBSycYccekKNqw-UKF==,1369474941778185308/repo is 3224347 2007-08-30 08:20:19,667 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,QhG8ZBSycYccekKNqw-UKF==,1369474941778185308 is 3224348 2007-08-30 08:20:19,668 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,QhG8ZBSycYccekKNqw-UKF==,1369474941778185308 available 2007-08-30 08:20:33,360 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo needs compaction 2007-08-30 08:20:33,360 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 2007-08-30 08:20:33,360 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:20:33,361 WARN org.apache.hadoop.hbase.HStore: Cleaning up a previous incomplete compaction at /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:20:39,555 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,o4Dadx140-iii3fJxp8HE-==,3823728776793780913, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:39,558 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,o4Dadx140-iii3fJxp8HE-==,3823728776793780913/repo (no reconstruction log) 2007-08-30 08:20:39,564 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,o4Dadx140-iii3fJxp8HE-==,3823728776793780913/repo is 3201392 2007-08-30 08:20:39,569 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,o4Dadx140-iii3fJxp8HE-==,3823728776793780913 is 3201393 2007-08-30 08:20:39,569 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,o4Dadx140-iii3fJxp8HE-==,3823728776793780913 available 2007-08-30 08:20:39,570 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,oJkLYnKh_k-zaTS4n2ABzk==,4715808464160863710, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:39,571 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,oJkLYnKh_k-zaTS4n2ABzk==,4715808464160863710/repo (no reconstruction log) 2007-08-30 08:20:39,576 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,oJkLYnKh_k-zaTS4n2ABzk==,4715808464160863710/repo is 3201392 2007-08-30 08:20:39,581 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,oJkLYnKh_k-zaTS4n2ABzk==,4715808464160863710 is 3201393 2007-08-30 08:20:39,582 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,oJkLYnKh_k-zaTS4n2ABzk==,4715808464160863710 available 2007-08-30 08:20:39,582 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,uVn11DWnuz4CU_I5a5OzJk==,6346305352248587804, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:39,583 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,uVn11DWnuz4CU_I5a5OzJk==,6346305352248587804/repo (no reconstruction log) 2007-08-30 08:20:39,593 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,uVn11DWnuz4CU_I5a5OzJk==,6346305352248587804/repo is 3223103 2007-08-30 08:20:39,600 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,uVn11DWnuz4CU_I5a5OzJk==,6346305352248587804 is 3223104 2007-08-30 08:20:39,601 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,uVn11DWnuz4CU_I5a5OzJk==,6346305352248587804 available 2007-08-30 08:20:39,601 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,ukF4NlM6zxscr-ueoZSiIV==,8463521469634242507, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:20:39,602 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,ukF4NlM6zxscr-ueoZSiIV==,8463521469634242507/repo (no reconstruction log) 2007-08-30 08:20:39,612 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,ukF4NlM6zxscr-ueoZSiIV==,8463521469634242507/repo is 3223103 2007-08-30 08:20:39,619 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,ukF4NlM6zxscr-ueoZSiIV==,8463521469634242507 is 3223104 2007-08-30 08:20:39,620 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,ukF4NlM6zxscr-ueoZSiIV==,8463521469634242507 available 2007-08-30 08:21:56,002 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache flush for region hbaserepository,VMMu5er_EV9dU-3jvIxXFV==,8469046003161993005. Size 16.0m 2007-08-30 08:21:56,002 DEBUG org.apache.hadoop.hbase.HRegion: Snapshotted memcache for region hbaserepository,VMMu5er_EV9dU-3jvIxXFV==,8469046003161993005 with sequence id 3237995 and entries 13467 2007-08-30 08:21:58,071 DEBUG org.apache.hadoop.hbase.HStore: Added hbaserepository,VMMu5er_EV9dU-3jvIxXFV==,8469046003161993005/repo/8774301867559349279 with sequence id 3237995 and size 16.3m 2007-08-30 08:21:58,072 DEBUG org.apache.hadoop.hbase.HRegion: Finished memcache flush for region hbaserepository,VMMu5er_EV9dU-3jvIxXFV==,8469046003161993005 in 2070ms 2007-08-30 08:22:08,551 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 35661 2007-08-30 08:22:09,271 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.091 to get a new one 2007-08-30 08:22:09,273 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.092 2007-08-30 08:22:29,578 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:22:30,459 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo (no reconstruction log) 2007-08-30 08:22:30,633 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo is 3249790 2007-08-30 08:22:30,821 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874 is 3249791 2007-08-30 08:22:30,822 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874 available 2007-08-30 08:24:26,006 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache flush for region hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611. Size 16.0m 2007-08-30 08:24:26,007 DEBUG org.apache.hadoop.hbase.HRegion: Snapshotted memcache for region hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611 with sequence id 3272676 and entries 13737 2007-08-30 08:24:29,609 INFO org.apache.hadoop.hbase.HRegionServer: MSG_REGION_OPEN : regionname: hbaserepository,Sc9rDIpkLoF-ZR4UG2VYPV==,2511412843035356707, startKey: , tableDesc: {name: hbaserepository, families: {repo:=(repo:, max versions: 3, compression: NONE, in memory: false, max value length: 2147483647, bloom filter: none)}} 2007-08-30 08:24:29,618 DEBUG org.apache.hadoop.hbase.HStore: starting hbaserepository,Sc9rDIpkLoF-ZR4UG2VYPV==,2511412843035356707/repo (no reconstruction log) 2007-08-30 08:24:29,707 DEBUG org.apache.hadoop.hbase.HStore: maximum sequence id for hstore hbaserepository,Sc9rDIpkLoF-ZR4UG2VYPV==,2511412843035356707/repo is 3268948 2007-08-30 08:24:29,752 DEBUG org.apache.hadoop.hbase.HRegion: maximum sequence id for region hbaserepository,Sc9rDIpkLoF-ZR4UG2VYPV==,2511412843035356707 is 3268949 2007-08-30 08:24:29,753 INFO org.apache.hadoop.hbase.HRegion: region hbaserepository,Sc9rDIpkLoF-ZR4UG2VYPV==,2511412843035356707 available 2007-08-30 08:24:30,413 DEBUG org.apache.hadoop.hbase.HStore: Added hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611/repo/7525211162607004775 with sequence id 3272676 and size 16.2m 2007-08-30 08:24:30,413 DEBUG org.apache.hadoop.hbase.HRegion: Finished memcache flush for region hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611 in 4407ms 2007-08-30 08:25:29,284 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33598 2007-08-30 08:25:30,202 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.092 to get a new one 2007-08-30 08:25:30,205 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.093 2007-08-30 08:25:33,525 ERROR org.apache.hadoop.hbase.HRegionServer: What happens if this fails? Are we toast? org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo/mapfiles/-1/data for DFSClient_1510910802 on client 208.76.44.141 because current leaseholder is trying to recreate file. at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:739) at org.apache.hadoop.dfs.NameNode.create(NameNode.java:307) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) at org.apache.hadoop.ipc.Client.call(Client.java:470) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1425) at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:370) at org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:127) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:272) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:676) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:165) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:127) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:107) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:92) at org.apache.hadoop.hbase.HStoreFile$BloomFilterMapFile$Writer.(HStoreFile.java:885) at org.apache.hadoop.hbase.HStoreFile.getWriter(HStoreFile.java:959) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:620) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:558) at org.apache.hadoop.hbase.HStore.compact(HStore.java:553) at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:699) at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.run(HRegionServer.java:169) at java.lang.Thread.run(Thread.java:619) 2007-08-30 08:25:33,559 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo needs compaction 2007-08-30 08:25:33,559 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874 2007-08-30 08:25:33,559 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo 2007-08-30 08:25:39,143 DEBUG org.apache.hadoop.hbase.HStore: moving hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo/-1 in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir to hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874/repo/2714061979485440599 in /bfd/hadoop-stack-data/tmp/hbase 2007-08-30 08:25:39,244 INFO org.apache.hadoop.hbase.HRegion: compaction completed on region hbaserepository,QDYLSmCLmUOpp8gIFTgESk==,3862894413685740874. Took 5sec 2007-08-30 08:25:39,248 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo needs compaction 2007-08-30 08:25:39,248 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 2007-08-30 08:25:39,248 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:26:20,207 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 32259 2007-08-30 08:26:20,847 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.093 to get a new one 2007-08-30 08:26:20,849 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.094 2007-08-30 08:27:20,852 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33618 2007-08-30 08:27:22,059 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.094 to get a new one 2007-08-30 08:27:22,061 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.095 2007-08-30 08:28:12,062 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33042 2007-08-30 08:28:13,483 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.095 to get a new one 2007-08-30 08:28:13,485 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.096 2007-08-30 08:29:03,701 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 32334 2007-08-30 08:29:04,478 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.096 to get a new one 2007-08-30 08:29:04,480 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.097 2007-08-30 08:29:54,482 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30348 2007-08-30 08:29:55,132 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.097 to get a new one 2007-08-30 08:29:55,134 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.098 2007-08-30 08:30:39,444 ERROR org.apache.hadoop.hbase.HRegionServer: What happens if this fails? Are we toast? org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo/mapfiles/-1/data for DFSClient_1510910802 on client 208.76.44.141 because current leaseholder is trying to recreate file. at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:739) at org.apache.hadoop.dfs.NameNode.create(NameNode.java:307) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) at org.apache.hadoop.ipc.Client.call(Client.java:470) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1425) at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:370) at org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:127) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:272) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:676) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:165) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:127) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:107) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:92) at org.apache.hadoop.hbase.HStoreFile$BloomFilterMapFile$Writer.(HStoreFile.java:885) at org.apache.hadoop.hbase.HStoreFile.getWriter(HStoreFile.java:959) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:620) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:558) at org.apache.hadoop.hbase.HStore.compact(HStore.java:553) at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:699) at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.run(HRegionServer.java:169) at java.lang.Thread.run(Thread.java:619) 2007-08-30 08:30:39,478 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo needs compaction 2007-08-30 08:30:39,478 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 2007-08-30 08:30:39,478 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:30:45,137 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30807 2007-08-30 08:30:46,413 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.098 to get a new one 2007-08-30 08:30:46,415 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.099 2007-08-30 08:31:46,417 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 35829 2007-08-30 08:31:47,163 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.099 to get a new one 2007-08-30 08:31:47,165 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.100 2007-08-30 08:32:47,209 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33144 2007-08-30 08:32:48,392 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.100 to get a new one 2007-08-30 08:32:48,395 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.101 2007-08-30 08:33:48,397 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33253 2007-08-30 08:33:49,606 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.101 to get a new one 2007-08-30 08:33:49,611 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.102 2007-08-30 08:34:49,615 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33876 2007-08-30 08:34:50,448 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.102 to get a new one 2007-08-30 08:34:50,450 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.103 2007-08-30 08:35:39,677 ERROR org.apache.hadoop.hbase.HRegionServer: What happens if this fails? Are we toast? org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo/mapfiles/-1/data for DFSClient_1510910802 on client 208.76.44.141 because current leaseholder is trying to recreate file. at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:739) at org.apache.hadoop.dfs.NameNode.create(NameNode.java:307) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) at org.apache.hadoop.ipc.Client.call(Client.java:470) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1425) at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:370) at org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:127) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:272) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:676) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:165) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:127) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:107) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:92) at org.apache.hadoop.hbase.HStoreFile$BloomFilterMapFile$Writer.(HStoreFile.java:885) at org.apache.hadoop.hbase.HStoreFile.getWriter(HStoreFile.java:959) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:620) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:558) at org.apache.hadoop.hbase.HStore.compact(HStore.java:553) at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:699) at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.run(HRegionServer.java:169) at java.lang.Thread.run(Thread.java:619) 2007-08-30 08:35:39,708 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo needs compaction 2007-08-30 08:35:39,708 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 2007-08-30 08:35:39,708 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:35:50,453 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 32331 2007-08-30 08:35:51,993 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.103 to get a new one 2007-08-30 08:35:51,995 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.104 2007-08-30 08:36:41,997 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30645 2007-08-30 08:36:43,093 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.104 to get a new one 2007-08-30 08:36:43,095 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.105 2007-08-30 08:37:43,099 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 34581 2007-08-30 08:37:44,299 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.105 to get a new one 2007-08-30 08:37:44,301 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.106 2007-08-30 08:38:44,303 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 31896 2007-08-30 08:38:45,511 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.106 to get a new one 2007-08-30 08:38:45,513 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.107 2007-08-30 08:39:45,518 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 33600 2007-08-30 08:39:46,801 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.107 to get a new one 2007-08-30 08:39:46,804 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.108 2007-08-30 08:40:39,952 ERROR org.apache.hadoop.hbase.HRegionServer: What happens if this fails? Are we toast? org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo/mapfiles/-1/data for DFSClient_1510910802 on client 208.76.44.141 because current leaseholder is trying to recreate file. at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:739) at org.apache.hadoop.dfs.NameNode.create(NameNode.java:307) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) at org.apache.hadoop.ipc.Client.call(Client.java:470) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:165) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.(DFSClient.java:1425) at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:370) at org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:127) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:272) at org.apache.hadoop.io.SequenceFile$Writer.(SequenceFile.java:676) at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:165) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:127) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:107) at org.apache.hadoop.io.MapFile$Writer.(MapFile.java:92) at org.apache.hadoop.hbase.HStoreFile$BloomFilterMapFile$Writer.(HStoreFile.java:885) at org.apache.hadoop.hbase.HStoreFile.getWriter(HStoreFile.java:959) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:620) at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:558) at org.apache.hadoop.hbase.HStore.compact(HStore.java:553) at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:699) at org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.run(HRegionServer.java:169) at java.lang.Thread.run(Thread.java:619) 2007-08-30 08:40:39,987 DEBUG org.apache.hadoop.hbase.HRegion: hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo needs compaction 2007-08-30 08:40:39,987 INFO org.apache.hadoop.hbase.HRegion: starting compaction on region hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947 2007-08-30 08:40:39,987 DEBUG org.apache.hadoop.hbase.HStore: started compaction of 3 files in /bfd/hadoop-stack-data/tmp/hbase/compaction.dir/hregion_hbaserepository,QT7BsplvgfJ2WPtnx3tbzk==,6182803324808354947/repo 2007-08-30 08:40:46,808 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30384 2007-08-30 08:40:47,417 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.108 to get a new one 2007-08-30 08:40:47,419 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.109 2007-08-30 08:41:47,421 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30300 2007-08-30 08:41:48,609 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.109 to get a new one 2007-08-30 08:41:48,611 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.110 2007-08-30 08:43:24,264 INFO org.apache.hadoop.hbase.HRegionServer: Rolling hlog. Number of entries: 30135 2007-08-30 08:43:24,847 DEBUG org.apache.hadoop.hbase.HLog: Closing current log writer /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.110 to get a new one 2007-08-30 08:43:24,849 DEBUG org.apache.hadoop.hbase.HLog: new log writer created at /bfd/hadoop-stack-data/tmp/hbase/log_208.76.44.141_60010/hlog.dat.111 2007-08-30 08:45:22,648 DEBUG org.apache.hadoop.hbase.HRegionServer: Got call server startup message 2007-08-30 08:45:22,648 DEBUG org.apache.hadoop.hbase.HRegionServer: closing region hbaserepository,-VSnfOBmHb04-5fzTBoSik==,7998154051278805764 2007-08-30 08:45:22,648 DEBUG org.apache.hadoop.hbase.HRegion: Started memcache flush for region hbaserepository,-VSnfOBmHb04-5fzTBoSik==,7998154051278805764. Size 10.3m 2007-08-30 08:45:22,648 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 60010, call batchUpdate(hbaserepository,bCQvHi6e6WBD42Bjrl5cNV==,6428295235976716263, 1188463436000, org.apache.hadoop.hbase.io.BatchUpdate@c73bfb) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,bCQvHi6e6WBD42Bjrl5cNV==,6428295235976716263 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,bCQvHi6e6WBD42Bjrl5cNV==,6428295235976716263 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:45:30,718 DEBUG org.apache.hadoop.hbase.HRegion: Snapshotted memcache for region hbaserepository,-VSnfOBmHb04-5fzTBoSik==,7998154051278805764 with sequence id 3862789 and entries 9795 2007-08-30 08:45:40,550 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 60010, call batchUpdate(hbaserepository,wEMiW-AU_mqMdDH7E5RsiF==,8940475657007858513, 1188463436000, org.apache.hadoop.hbase.io.BatchUpdate@2f4356) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,wEMiW-AU_mqMdDH7E5RsiF==,8940475657007858513 2007-08-30 08:46:09,483 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 60010, call batchUpdate(hbaserepository,JKUoseNSKw3jbbLFjbvbu-==,8262090525775564947, 1188463436000, org.apache.hadoop.hbase.io.BatchUpdate@93370a) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,JKUoseNSKw3jbbLFjbvbu-==,8262090525775564947 2007-08-30 08:46:02,262 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 60010, call batchUpdate(hbaserepository,Bd7vwtr8WeN_UMPru2Tnsk==,2812793047156847230, 1188463426000, org.apache.hadoop.hbase.io.BatchUpdate@cf2e49) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,Bd7vwtr8WeN_UMPru2Tnsk==,2812793047156847230 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,Bd7vwtr8WeN_UMPru2Tnsk==,2812793047156847230 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:45:57,445 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 60010, call batchUpdate(hbaserepository,TMIfpmgexaZ1JovoV0olqk==,36307571716794938, 1188463426000, org.apache.hadoop.hbase.io.BatchUpdate@1540b40) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,TMIfpmgexaZ1JovoV0olqk==,36307571716794938 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,TMIfpmgexaZ1JovoV0olqk==,36307571716794938 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:45:57,445 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 60010, call batchUpdate(hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611, 1188463426000, org.apache.hadoop.hbase.io.BatchUpdate@969ebb) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,QwzRj4dQS9jsSTsohL8W-k==,1907718153167081611 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:48:47,896 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 60010, call batchUpdate(hbaserepository,CNLWSL_lmDzjO3zw-7jEEF==,5823065058369275234, 1188463431000, org.apache.hadoop.hbase.io.BatchUpdate@27e2d8) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,CNLWSL_lmDzjO3zw-7jEEF==,5823065058369275234 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,CNLWSL_lmDzjO3zw-7jEEF==,5823065058369275234 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:47:41,463 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60010, call batchUpdate(hbaserepository,jJhUDp162tyRDt3XivQI2F==,7979273270319385493, 1188463431000, org.apache.hadoop.hbase.io.BatchUpdate@1ead510) from 208.76.44.139:49917: error: org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,jJhUDp162tyRDt3XivQI2F==,7979273270319385493 org.apache.hadoop.hbase.NotServingRegionException: hbaserepository,jJhUDp162tyRDt3XivQI2F==,7979273270319385493 at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1227) at org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1199) at org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1136) at org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:994) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:340) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:566) 2007-08-30 08:47:36,637 WARN org.apache.hadoop.ipc.Server: Out of Memory in server select java.lang.OutOfMemoryError: Java heap space