Thu Jul 31 13:19:00 PDT 2008 Starting regionserver on hb2.streamy.com java version "1.6.0" OpenJDK Runtime Environment (build 1.6.0-b09) OpenJDK 64-Bit Server VM (build 1.6.0-b09, mixed mode) ulimit -n 65535 2008-07-31 13:19:01,104 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=HRegionServer, port=60020 2008-07-31 13:19:01,180 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at mb0:60000 that we are up 2008-07-31 13:19:01,389 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.regionserver.address=72.34.249.212 2008-07-31 13:19:01,389 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: fs.default.name=hdfs://mb0:9000/hbase 2008-07-31 13:19:01,390 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master: hbase.rootdir=hdfs://mb0:9000/hbase 2008-07-31 13:19:01,488 DEBUG org.apache.hadoop.hbase.regionserver.HRegionServer: Log dir hdfs://mb0:9000/hbase/log_72.34.249.212_1217535541159_60020 2008-07-31 13:19:01,566 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217535541536 2008-07-31 13:19:01,610 INFO org.mortbay.util.Credential: Checking Resource aliases 2008-07-31 13:19:01,642 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4 2008-07-31 13:19:01,643 INFO org.mortbay.util.Container: Started HttpContext[/static,/static] 2008-07-31 13:19:01,643 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs] 2008-07-31 13:19:01,888 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@3c3a1834 2008-07-31 13:19:01,916 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/] 2008-07-31 13:19:01,918 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:60030 2008-07-31 13:19:01,918 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@50269997 2008-07-31 13:19:01,918 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 2008-07-31 13:19:01,918 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 60020: starting 2008-07-31 13:19:01,920 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 60020: starting 2008-07-31 13:19:01,920 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 60020: starting 2008-07-31 13:19:01,920 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 60020: starting 2008-07-31 13:19:01,920 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 60020: starting 2008-07-31 13:19:01,920 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 60020: starting 2008-07-31 13:19:01,921 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: HRegionServer started at: 72.34.249.212:60020 2008-07-31 13:20:56,034 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: threads,,1217535654882 2008-07-31 13:20:56,035 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: threads,,1217535654882 2008-07-31 13:20:56,109 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Found ROOT REGION => {NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED => 70236052, TABLE => {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true', FAMILIES => [{NAME => 'info', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '1', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}} 2008-07-31 13:20:56,119 DEBUG org.apache.hadoop.hbase.RegionHistorian: Onlined 2008-07-31 13:20:56,127 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region threads,,1217535654882/580569749 2008-07-31 13:20:56,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region threads,,1217535654882 is 0 2008-07-31 13:20:56,155 INFO org.apache.hadoop.hbase.regionserver.HRegion: region threads,,1217535654882/580569749 available 2008-07-31 13:20:56,155 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: threads,,1217535654882 2008-07-31 13:20:56,155 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region threads,,1217535654882 2008-07-31 13:20:56,157 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region threads,,1217535654882 in 0sec 2008-07-31 13:49:01,570 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 0 2008-07-31 13:49:01,599 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.0 2008-07-31 13:49:01,605 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217537341600 2008-07-31 13:49:01,605 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Last sequence written is empty. Deleting all old hlogs 2008-07-31 13:49:01,605 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.0 whose highest sequence/edit id is -1 2008-07-31 14:19:01,609 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 0 2008-07-31 14:19:01,626 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217535541536 2008-07-31 14:19:01,633 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217539141626 2008-07-31 14:19:01,633 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Last sequence written is empty. Deleting all old hlogs 2008-07-31 14:19:01,633 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217535541536 whose highest sequence/edit id is -1 2008-07-31 14:49:01,638 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 712 2008-07-31 14:49:01,650 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217537341600 2008-07-31 14:49:01,654 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217540941650 2008-07-31 14:51:11,597 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region threads,,1217535654882. Current region memcache size 243.0k 2008-07-31 14:51:11,618 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library 2008-07-31 14:51:11,619 INFO org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library 2008-07-31 14:51:11,719 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/threads/580569749/content/mapfiles/905965078069540699 with 326 entries, sequence id 712, data size 27.4k, file size 32.3k 2008-07-31 14:51:11,813 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/threads/580569749/posts/mapfiles/3583203531811787404 with 386 entries, sequence id 712, data size 215.6k, file size 222.2k 2008-07-31 14:51:11,814 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region threads,,1217535654882 in 218ms, sequence id=712, compaction requested=false 2008-07-31 15:19:01,658 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 1 2008-07-31 15:19:01,671 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217539141626 2008-07-31 15:19:01,673 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217542741671 2008-07-31 15:19:01,673 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Last sequence written is empty. Deleting all old hlogs 2008-07-31 15:19:01,673 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217537341600 whose highest sequence/edit id is 711 2008-07-31 15:19:01,674 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217539141626 whose highest sequence/edit id is 712 2008-07-31 15:30:57,247 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:30:57,248 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 15:30:57,248 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:30:57,248 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296/572594574 2008-07-31 15:30:57,281 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/savedby/info/255876046741365226.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,283 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 572594574/savedby, max sequence id 110420 2008-07-31 15:30:57,302 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/readby/info/6761703316498463736.1341247330, isReference=true, sequence id=546972 2008-07-31 15:30:57,303 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 572594574/readby, max sequence id 546972 2008-07-31 15:30:57,321 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/receivedby/info/50952989531154315.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,322 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 572594574/receivedby, max sequence id 110420 2008-07-31 15:30:57,360 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/sentby/info/3669836667077479368.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,361 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 572594574/sentby, max sequence id 110420 2008-07-31 15:30:57,380 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/content/info/452338376198752884.1341247330, isReference=true, sequence id=3857865 2008-07-31 15:30:57,389 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/572594574/content/info/8750413624746122519.1341247330, isReference=true, sequence id=3854718 2008-07-31 15:30:57,390 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 572594574/content, max sequence id 3857865 2008-07-31 15:30:57,407 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 is 3857866 2008-07-31 15:30:57,408 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296/572594574 available 2008-07-31 15:30:57,408 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:30:57,408 DEBUG org.apache.hadoop.hbase.regionserver.HLog: changing sequence number from 713 to 3857866 2008-07-31 15:30:57,408 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:30:57,408 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 15:30:57,435 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296/853828343 2008-07-31 15:30:57,436 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [572594574/content/8750413624746122519-1341247330/8510314036402829536/bottom, 572594574/content/452338376198752884-1341247330/1070777688983551195/bottom] 572594574/content 2008-07-31 15:30:57,500 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 2 files into /hbase/items/compaction.dir/572594574/content/mapfiles/4687843736761630366 2008-07-31 15:30:57,501 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/savedby/info/2676158957210538468.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,503 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 853828343/savedby, max sequence id 110420 2008-07-31 15:30:57,524 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/readby/info/7780712244651295436.1341247330, isReference=true, sequence id=546972 2008-07-31 15:30:57,525 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 853828343/readby, max sequence id 546972 2008-07-31 15:30:57,540 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/receivedby/info/2560274113410690074.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,541 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 853828343/receivedby, max sequence id 110420 2008-07-31 15:30:57,575 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/sentby/info/3844179586805000603.1341247330, isReference=true, sequence id=110420 2008-07-31 15:30:57,575 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 853828343/sentby, max sequence id 110420 2008-07-31 15:30:57,593 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/content/info/1475699774493439732.1341247330, isReference=true, sequence id=3857865 2008-07-31 15:30:57,600 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/853828343/content/info/2459821554546718947.1341247330, isReference=true, sequence id=3854718 2008-07-31 15:30:57,601 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 853828343/content, max sequence id 3857865 2008-07-31 15:30:57,661 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 is 3857866 2008-07-31 15:30:57,662 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296/853828343 available 2008-07-31 15:30:57,662 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 15:31:05,218 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/content/mapfiles/4687843736761630366 to /hbase/items/572594574/content/mapfiles/5490391791624899457 2008-07-31 15:31:05,320 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/content store size is 135.7m 2008-07-31 15:31:05,321 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [572594574/readby/6761703316498463736-1341247330/2457372037986878574/bottom] 572594574/readby 2008-07-31 15:31:05,330 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/572594574/readby/mapfiles/7087055545672032654 2008-07-31 15:31:06,655 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/readby/mapfiles/7087055545672032654 to /hbase/items/572594574/readby/mapfiles/2032814401034614781 2008-07-31 15:31:06,681 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/readby store size is 18.3m 2008-07-31 15:31:06,682 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [572594574/savedby/255876046741365226-1341247330/907837599844339120/bottom] 572594574/savedby 2008-07-31 15:31:06,691 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/572594574/savedby/mapfiles/7379154288858709596 2008-07-31 15:31:06,788 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/savedby/mapfiles/7379154288858709596 to /hbase/items/572594574/savedby/mapfiles/5168359651499534413 2008-07-31 15:31:06,807 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/savedby store size is 797.5k 2008-07-31 15:31:06,808 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [572594574/sentby/3669836667077479368-1341247330/7874730748448724920/bottom] 572594574/sentby 2008-07-31 15:31:06,815 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/572594574/sentby/mapfiles/8488463559716739497 2008-07-31 15:31:06,880 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/sentby/mapfiles/8488463559716739497 to /hbase/items/572594574/sentby/mapfiles/3829953817800611234 2008-07-31 15:31:06,901 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/sentby store size is 301.2k 2008-07-31 15:31:06,901 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [572594574/receivedby/50952989531154315-1341247330/4961047972238348388/bottom] 572594574/receivedby 2008-07-31 15:31:06,909 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/572594574/receivedby/mapfiles/5359771141435910745 2008-07-31 15:31:06,989 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/receivedby/mapfiles/5359771141435910745 to /hbase/items/572594574/receivedby/mapfiles/457274427669556234 2008-07-31 15:31:07,006 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/receivedby store size is 311.4k 2008-07-31 15:31:07,009 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 in 9sec 2008-07-31 15:31:07,009 DEBUG org.apache.hadoop.hbase.util.SoftSortedMap: Reference for key has been cleared. 2008-07-31 15:31:07,012 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Found ROOT REGION => {NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED => 70236052, TABLE => {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true', FAMILIES => [{NAME => 'info', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '1', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}} 2008-07-31 15:31:07,029 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 15:31:07,029 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [853828343/content/2459821554546718947-1341247330/8510314036402829536/top, 853828343/content/1475699774493439732-1341247330/1070777688983551195/top] 853828343/content 2008-07-31 15:31:07,046 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 2 files into /hbase/items/compaction.dir/853828343/content/mapfiles/1014263152102586973 2008-07-31 15:31:14,085 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/content/mapfiles/1014263152102586973 to /hbase/items/853828343/content/mapfiles/7150052340741078436 2008-07-31 15:31:14,119 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/content store size is 137.4m 2008-07-31 15:31:14,119 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [853828343/readby/7780712244651295436-1341247330/2457372037986878574/top] 853828343/readby 2008-07-31 15:31:14,127 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/853828343/readby/mapfiles/9012471484945033816 2008-07-31 15:31:15,536 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/readby/mapfiles/9012471484945033816 to /hbase/items/853828343/readby/mapfiles/2967318044136786382 2008-07-31 15:31:15,560 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/readby store size is 18.6m 2008-07-31 15:31:15,560 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [853828343/savedby/2676158957210538468-1341247330/907837599844339120/top] 853828343/savedby 2008-07-31 15:31:15,567 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/853828343/savedby/mapfiles/8073635732227385118 2008-07-31 15:31:15,667 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/savedby/mapfiles/8073635732227385118 to /hbase/items/853828343/savedby/mapfiles/8842526882507793138 2008-07-31 15:31:15,687 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/savedby store size is 811.0k 2008-07-31 15:31:15,687 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [853828343/sentby/3844179586805000603-1341247330/7874730748448724920/top] 853828343/sentby 2008-07-31 15:31:15,696 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/853828343/sentby/mapfiles/394810255270348630 2008-07-31 15:31:15,775 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/sentby/mapfiles/394810255270348630 to /hbase/items/853828343/sentby/mapfiles/8867356411558358681 2008-07-31 15:31:15,793 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/sentby store size is 319.8k 2008-07-31 15:31:15,793 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [853828343/receivedby/2560274113410690074-1341247330/4961047972238348388/top] 853828343/receivedby 2008-07-31 15:31:15,800 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/853828343/receivedby/mapfiles/6161062065298085677 2008-07-31 15:31:15,862 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/receivedby/mapfiles/6161062065298085677 to /hbase/items/853828343/receivedby/mapfiles/2219321021993316861 2008-07-31 15:31:15,878 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/receivedby store size is 329.3k 2008-07-31 15:31:15,881 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 in 8sec 2008-07-31 15:31:29,249 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:29,257 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217540941650 2008-07-31 15:31:29,260 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543489257 2008-07-31 15:31:31,878 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:31,886 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217542741671 2008-07-31 15:31:31,888 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543491886 2008-07-31 15:31:34,184 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:34,193 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543489257 2008-07-31 15:31:34,195 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543494193 2008-07-31 15:31:35,703 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:35,710 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543491886 2008-07-31 15:31:35,712 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543495710 2008-07-31 15:31:37,474 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:37,481 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543494193 2008-07-31 15:31:37,483 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543497481 2008-07-31 15:31:40,045 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:40,052 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543495710 2008-07-31 15:31:40,054 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543500052 2008-07-31 15:31:42,727 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:31:42,733 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543497481 2008-07-31 15:31:42,734 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543502733 2008-07-31 15:32:05,076 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:05,122 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543500052 2008-07-31 15:32:05,124 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543525122 2008-07-31 15:32:07,967 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:07,974 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543502733 2008-07-31 15:32:07,976 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543527974 2008-07-31 15:32:10,496 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:10,542 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543525122 2008-07-31 15:32:10,544 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543530542 2008-07-31 15:32:54,495 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:54,505 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543527974 2008-07-31 15:32:54,506 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543574505 2008-07-31 15:32:57,009 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:57,055 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543530542 2008-07-31 15:32:57,056 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543577055 2008-07-31 15:32:59,430 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:32:59,439 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543574505 2008-07-31 15:32:59,441 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543579439 2008-07-31 15:33:01,840 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:01,848 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543577055 2008-07-31 15:33:01,851 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543581849 2008-07-31 15:33:03,114 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296. Current region memcache size 64.0m 2008-07-31 15:33:05,004 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:05,707 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/853828343/content/mapfiles/899771229557496049 with 325226 entries, sequence id 4293092, data size 64.0m, file size 68.8m 2008-07-31 15:33:05,707 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 in 2593ms, sequence id=4293092, compaction requested=false 2008-07-31 15:33:05,724 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543579439 2008-07-31 15:33:05,727 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543585725 2008-07-31 15:33:05,727 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Found 2 logs to remove using oldest outstanding seqnum of 3930059 from region [B@10dc942b 2008-07-31 15:33:05,727 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217540941650 whose highest sequence/edit id is 3887873 2008-07-31 15:33:05,728 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217542741671 whose highest sequence/edit id is 3917881 2008-07-31 15:33:08,254 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:08,263 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543581849 2008-07-31 15:33:08,264 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543588263 2008-07-31 15:33:10,590 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:10,597 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543585725 2008-07-31 15:33:10,598 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543590597 2008-07-31 15:33:12,945 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:12,953 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543588263 2008-07-31 15:33:12,954 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543592953 2008-07-31 15:33:33,778 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:33,786 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543590597 2008-07-31 15:33:33,788 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543613786 2008-07-31 15:33:36,179 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:36,186 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543592953 2008-07-31 15:33:36,188 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543616186 2008-07-31 15:33:38,560 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:33:38,569 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543613786 2008-07-31 15:33:38,570 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543618569 2008-07-31 15:34:22,007 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:22,015 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543616186 2008-07-31 15:34:22,016 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543662015 2008-07-31 15:34:24,708 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:24,715 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543618569 2008-07-31 15:34:24,716 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543664715 2008-07-31 15:34:27,327 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:27,333 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543662015 2008-07-31 15:34:27,334 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543667333 2008-07-31 15:34:29,954 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:30,000 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543664715 2008-07-31 15:34:30,002 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543670001 2008-07-31 15:34:35,805 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:35,812 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543667333 2008-07-31 15:34:35,813 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543675812 2008-07-31 15:34:38,239 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:38,246 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543670001 2008-07-31 15:34:38,248 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543678246 2008-07-31 15:34:40,658 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:40,665 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543675812 2008-07-31 15:34:40,667 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543680665 2008-07-31 15:34:42,843 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296. Current region memcache size 64.0m 2008-07-31 15:34:43,091 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:44,651 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/572594574/content/mapfiles/209157333742621660 with 324962 entries, sequence id 4732829, data size 64.0m, file size 68.8m 2008-07-31 15:34:44,651 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 in 1808ms, sequence id=4732829, compaction requested=false 2008-07-31 15:34:44,696 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543678246 2008-07-31 15:34:44,699 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543684696 2008-07-31 15:34:44,700 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Found 16 logs to remove using oldest outstanding seqnum of 4407867 from region [B@3541d281 2008-07-31 15:34:44,700 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543489257 whose highest sequence/edit id is 3947889 2008-07-31 15:34:44,702 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543491886 whose highest sequence/edit id is 3977897 2008-07-31 15:34:44,704 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543494193 whose highest sequence/edit id is 4007905 2008-07-31 15:34:44,706 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543495710 whose highest sequence/edit id is 4037913 2008-07-31 15:34:44,708 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543497481 whose highest sequence/edit id is 4067921 2008-07-31 15:34:44,710 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543500052 whose highest sequence/edit id is 4097929 2008-07-31 15:34:44,712 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543502733 whose highest sequence/edit id is 4127937 2008-07-31 15:34:44,713 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543525122 whose highest sequence/edit id is 4157945 2008-07-31 15:34:44,714 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543527974 whose highest sequence/edit id is 4187953 2008-07-31 15:34:44,715 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543530542 whose highest sequence/edit id is 4217961 2008-07-31 15:34:44,717 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543574505 whose highest sequence/edit id is 4247969 2008-07-31 15:34:44,719 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543577055 whose highest sequence/edit id is 4277977 2008-07-31 15:34:44,732 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543579439 whose highest sequence/edit id is 4315928 2008-07-31 15:34:44,734 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543581849 whose highest sequence/edit id is 4345936 2008-07-31 15:34:44,736 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543585725 whose highest sequence/edit id is 4375944 2008-07-31 15:34:44,737 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543588263 whose highest sequence/edit id is 4405952 2008-07-31 15:34:58,364 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:34:58,373 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543680665 2008-07-31 15:34:58,374 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543698373 2008-07-31 15:35:00,853 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:35:00,859 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543684696 2008-07-31 15:35:00,860 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543700859 2008-07-31 15:35:03,243 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:35:03,249 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543698373 2008-07-31 15:35:03,251 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543703249 2008-07-31 15:35:03,975 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 3 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:35:39,464 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 5 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:35:53,621 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:35:53,629 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543700859 2008-07-31 15:35:53,631 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543753629 2008-07-31 15:35:56,183 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:35:56,190 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543703249 2008-07-31 15:35:56,192 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543756190 2008-07-31 15:35:58,507 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:35:58,514 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543753629 2008-07-31 15:35:58,515 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543758514 2008-07-31 15:36:14,001 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 4 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:36:49,493 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 6 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:36:55,683 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:36:55,691 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543756190 2008-07-31 15:36:55,694 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543815691 2008-07-31 15:36:58,332 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:36:58,397 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543758514 2008-07-31 15:36:58,398 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543818397 2008-07-31 15:37:01,079 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:37:01,087 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543815691 2008-07-31 15:37:01,088 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543821087 2008-07-31 15:37:03,849 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:37:03,856 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543818397 2008-07-31 15:37:03,857 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543823856 2008-07-31 15:37:24,027 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 0 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:37:59,175 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:37:59,183 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543821087 2008-07-31 15:37:59,185 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543879184 2008-07-31 15:37:59,519 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 2 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:38:01,880 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:38:01,887 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543823856 2008-07-31 15:38:01,888 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543881887 2008-07-31 15:38:04,683 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:38:04,690 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543879184 2008-07-31 15:38:04,692 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543884691 2008-07-31 15:38:07,024 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296. Current region memcache size 64.0m 2008-07-31 15:38:07,215 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 15:38:09,050 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/572594574/content/mapfiles/165049377686716198 with 323906 entries, sequence id 5155648, data size 64.0m, file size 68.8m 2008-07-31 15:38:09,050 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 in 2026ms, sequence id=5155648, compaction requested=true 2008-07-31 15:38:09,050 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:09,050 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:09,051 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [572594574/content/5490391791624899457, 572594574/content/209157333742621660, 572594574/content/165049377686716198] 572594574/content 2008-07-31 15:38:09,054 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction size of 572594574/content: 273.6m, skipped 0, 0 2008-07-31 15:38:09,125 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543881887 2008-07-31 15:38:09,128 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543889125 2008-07-31 15:38:09,144 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 3 files into /hbase/items/compaction.dir/572594574/content/mapfiles/5321734387639641998 2008-07-31 15:38:23,585 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/572594574/content/mapfiles/5321734387639641998 to /hbase/items/572594574/content/mapfiles/4322311413586672520 2008-07-31 15:38:23,646 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 572594574/content store size is 273.3m 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [572594574/readby/2032814401034614781] 572594574/readby 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [572594574/savedby/5168359651499534413] 572594574/savedby 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [572594574/sentby/3829953817800611234] 572594574/sentby 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [572594574/receivedby/457274427669556234] 572594574/receivedby 2008-07-31 15:38:23,649 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 15:38:23,651 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 in 14sec 2008-07-31 15:38:23,653 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,656 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,656 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,656 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,657 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,657 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296. Current region memcache size 2.6m 2008-07-31 15:38:23,761 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/572594574/content/mapfiles/6495859137972513446 with 11132 entries, sequence id 5166781, data size 2.6m, file size 2.7m 2008-07-31 15:38:23,761 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 in 104ms, sequence id=5166781, compaction requested=false 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/uclusters 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/covisit 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/content 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/clusters 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/readby 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/cfrecs 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/blooms 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/ranked 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/savedby 2008-07-31 15:38:23,762 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/sentby 2008-07-31 15:38:23,763 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 572594574/receivedby 2008-07-31 15:38:23,763 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 2008-07-31 15:38:23,957 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655/1564280864 2008-07-31 15:38:23,981 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/savedby/info/119841459483002582.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:23,981 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1564280864/savedby, max sequence id 110420 2008-07-31 15:38:23,993 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/readby/info/797940275383626254.572594574, isReference=true, sequence id=546972 2008-07-31 15:38:23,994 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1564280864/readby, max sequence id 546972 2008-07-31 15:38:24,006 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/receivedby/info/876039824146417957.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:24,006 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1564280864/receivedby, max sequence id 110420 2008-07-31 15:38:24,038 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/sentby/info/2948077811691085902.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:24,039 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1564280864/sentby, max sequence id 110420 2008-07-31 15:38:24,051 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/content/info/2583322844363598753.572594574, isReference=true, sequence id=5155648 2008-07-31 15:38:24,057 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1564280864/content/info/5792021966706888590.572594574, isReference=true, sequence id=5166781 2008-07-31 15:38:24,057 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 1564280864/content, max sequence id 5166781 2008-07-31 15:38:24,067 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 is 5166782 2008-07-31 15:38:24,068 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655/1564280864 available 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/uclusters 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/covisit 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/content 2008-07-31 15:38:24,068 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/clusters 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/readby 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/cfrecs 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/blooms 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/ranked 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/savedby 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/sentby 2008-07-31 15:38:24,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1564280864/receivedby 2008-07-31 15:38:24,069 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655 2008-07-31 15:38:24,070 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655/590792438 2008-07-31 15:38:24,090 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/savedby/info/2296212123918213448.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:24,091 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 590792438/savedby, max sequence id 110420 2008-07-31 15:38:24,105 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/readby/info/8749908172951496241.572594574, isReference=true, sequence id=546972 2008-07-31 15:38:24,105 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 590792438/readby, max sequence id 546972 2008-07-31 15:38:24,117 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/receivedby/info/4493317860263148476.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:24,118 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 590792438/receivedby, max sequence id 110420 2008-07-31 15:38:24,151 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/sentby/info/3391538397429867530.572594574, isReference=true, sequence id=110420 2008-07-31 15:38:24,151 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 590792438/sentby, max sequence id 110420 2008-07-31 15:38:24,164 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/content/info/1178653687141436412.572594574, isReference=true, sequence id=5155648 2008-07-31 15:38:24,169 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/590792438/content/info/2725253370086844330.572594574, isReference=true, sequence id=5166781 2008-07-31 15:38:24,170 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 590792438/content, max sequence id 5166781 2008-07-31 15:38:24,182 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 is 5166782 2008-07-31 15:38:24,183 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655/590792438 available 2008-07-31 15:38:24,183 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 2008-07-31 15:38:24,183 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/uclusters 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/covisit 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/content 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/clusters 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/readby 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/cfrecs 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/blooms 2008-07-31 15:38:24,184 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/ranked 2008-07-31 15:38:24,185 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/savedby 2008-07-31 15:38:24,185 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/sentby 2008-07-31 15:38:24,186 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 590792438/receivedby 2008-07-31 15:38:24,186 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,02407831-194c-4267-a385-6bac7005b11d,1217543903655 2008-07-31 15:38:24,187 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Cleaned up /hbase/items/572594574/splits true 2008-07-31 15:38:24,190 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Cache hit in table locations for row <> and tableName .META.: location server 72.34.249.217:60020, location region name .META.,,1 2008-07-31 15:38:24,194 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: region split, META updated, and report to master all successful. Old region=REGION => {NAME => 'items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296', STARTKEY => '01beddd6-813b-4f2b-ac48-a0cef395cb7e', ENDKEY => '8001eb31-98bb-4087-bd8d-e4b42805addb', ENCODED => 572594574, OFFLINE => true, SPLIT => true, TABLE => {{NAME => 'items', IS_ROOT => 'false', IS_META => 'false', FAMILIES => [{NAME => 'clusters', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'cfrecs', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'savedby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'readby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'receivedby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'ranked', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'blooms', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'covisit', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'uclusters', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'sentby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'content', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}, new regions: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543903655, items,02407831-194c-4267-a385-6bac7005b11d,1217543903655. Split took 0sec 2008-07-31 15:38:34,052 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 7 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:39:00,270 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 60020, call batchUpdate([B@17b4239f, row => 02c241b4-9d32-452d-8dab-247f4af693eb, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50861: error: org.apache.hadoop.hbase.NotServingRegionException: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 org.apache.hadoop.hbase.NotServingRegionException: items,01beddd6-813b-4f2b-ac48-a0cef395cb7e,1217543451296 at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:1436) at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1147) at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) 2008-07-31 15:39:09,547 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 8 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:39:44,079 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 9 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:40:19,574 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 1 on 60020' on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296: Memcache size 64.0m is >= than blocking 64.0m size 2008-07-31 15:49:09,130 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 1 2008-07-31 15:49:09,144 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543884691 2008-07-31 15:49:09,146 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217544549145 2008-07-31 16:03:09,060 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296. Current region memcache size 64.0m 2008-07-31 16:03:09,467 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 5 on 60020' 2008-07-31 16:03:09,478 INFO org.apache.hadoop.ipc.Server: Process Thread Dump: Discarding call batchUpdate([B@4e727e0e, row => c08408b4-b68c-433c-ba3f-d46d3ba73288, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815 34 active threads Thread 3958 (ResponseProcessor for block blk_7533464124404356942): State: RUNNABLE Blocked count: 37 Waited count: 0 Stack: sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:233) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:237) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:155) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:149) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:122) java.io.DataInputStream.readFully(DataInputStream.java:195) java.io.DataInputStream.readLong(DataInputStream.java:416) org.apache.hadoop.dfs.DFSClient$DFSOutputStream$ResponseProcessor.run(DFSClient.java:1982) Thread 3957 (Thread-3944): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1824) Thread 3956 (DataStreamer for file /hbase/items/853828343/content/mapfiles/2345206955841469254/data block blk_7533464124404356942): State: RUNNABLE Blocked count: 0 Waited count: 615 Stack: sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:233) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:237) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:155) org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:144) org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:105) java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) java.io.DataOutputStream.write(DataOutputStream.java:107) org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1880) Thread 3955 (IPC Client connection to mb0/72.34.249.200:9000): State: WAITING Blocked count: 1 Waited count: 5 Waiting on org.apache.hadoop.ipc.Client$Connection@7ce5bd Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:247) org.apache.hadoop.ipc.Client$Connection.run(Client.java:286) Thread 3646 (Thread-3633): State: TIMED_WAITING Blocked count: 0 Waited count: 841 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1824) Thread 45 (IPC Server handler 9 on 60020): State: TIMED_WAITING Blocked count: 512 Waited count: 14579 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 44 (IPC Server handler 8 on 60020): State: TIMED_WAITING Blocked count: 478 Waited count: 14581 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 43 (IPC Server handler 7 on 60020): State: TIMED_WAITING Blocked count: 541 Waited count: 14584 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 42 (IPC Server handler 6 on 60020): State: TIMED_WAITING Blocked count: 392 Waited count: 10496 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 41 (IPC Server handler 5 on 60020): State: RUNNABLE Blocked count: 331 Waited count: 9252 Stack: sun.management.ThreadImpl.getThreadInfo0(Native Method) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:165) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:141) org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:114) org.apache.hadoop.util.ReflectionUtils.logThreadInfo(ReflectionUtils.java:168) org.apache.hadoop.ipc.Server$Handler.run(Server.java:875) Thread 40 (IPC Server handler 4 on 60020): State: TIMED_WAITING Blocked count: 383 Waited count: 10499 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 39 (IPC Server handler 3 on 60020): State: TIMED_WAITING Blocked count: 366 Waited count: 9256 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 38 (IPC Server handler 2 on 60020): State: TIMED_WAITING Blocked count: 405 Waited count: 12425 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 37 (IPC Server handler 1 on 60020): State: TIMED_WAITING Blocked count: 524 Waited count: 14577 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 36 (IPC Server handler 0 on 60020): State: TIMED_WAITING Blocked count: 412 Waited count: 12160 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.HRegion.checkResources(HRegion.java:1425) org.apache.hadoop.hbase.regionserver.HRegion.batchUpdate(HRegion.java:1343) org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1151) sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:616) org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) Thread 12 (IPC Server listener on 60020): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:233) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.ipc.Server$Listener.run(Server.java:299) Thread 14 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:233) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:83) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.ipc.Server$Responder.run(Server.java:445) Thread 35 (SocketListener0-1): State: TIMED_WAITING Blocked count: 0 Waited count: 985 Stack: java.lang.Object.wait(Native Method) org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:522) Thread 34 (SocketListener0-0): State: TIMED_WAITING Blocked count: 1 Waited count: 985 Stack: java.lang.Object.wait(Native Method) org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:522) Thread 33 (Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.net.PlainSocketImpl.socketAccept(Native Method) java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:358) java.net.ServerSocket.implAccept(ServerSocket.java:470) java.net.ServerSocket.accept(ServerSocket.java:438) org.mortbay.util.ThreadedServer.acceptSocket(ThreadedServer.java:432) org.mortbay.util.ThreadedServer$Acceptor.run(ThreadedServer.java:631) Thread 32 (SessionScavenger): State: TIMED_WAITING Blocked count: 0 Waited count: 329 Stack: java.lang.Thread.sleep(Native Method) org.mortbay.jetty.servlet.AbstractSessionManager$SessionScavenger.run(AbstractSessionManager.java:587) Thread 15 (regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker): State: TIMED_WAITING Blocked count: 0 Waited count: 985 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2016) java.util.concurrent.DelayQueue.poll(DelayQueue.java:230) org.apache.hadoop.hbase.Leases.run(Leases.java:75) Thread 11 (regionserver/0:0:0:0:0:0:0:0:60020.worker): State: TIMED_WAITING Blocked count: 2 Waited count: 1242 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2016) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:423) org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:804) java.lang.Thread.run(Thread.java:636) Thread 9 (regionserver/0:0:0:0:0:0:0:0:60020.compactor): State: TIMED_WAITING Blocked count: 4886 Waited count: 1406 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2016) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:423) org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:76) Thread 8 (regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher): State: RUNNABLE Blocked count: 2343 Waited count: 1151 Stack: java.util.zip.CRC32.updateBytes(Native Method) java.util.zip.CRC32.update(CRC32.java:62) org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:107) org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86) org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:47) java.io.DataOutputStream.write(DataOutputStream.java:107) org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1023) org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:987) org.apache.hadoop.io.MapFile$Writer.append(MapFile.java:198) org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.append(HStoreFile.java:842) org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:615) org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:567) org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1054) org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:965) org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:173) org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:91) Thread 10 (regionserver/0:0:0:0:0:0:0:0:60020.logRoller): State: TIMED_WAITING Blocked count: 4 Waited count: 1296 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:63) Thread 30 (org.apache.hadoop.dfs.DFSClient$LeaseChecker@877ef83): State: TIMED_WAITING Blocked count: 0 Waited count: 10170 Stack: java.lang.Thread.sleep(Native Method) org.apache.hadoop.dfs.DFSClient$LeaseChecker.run(DFSClient.java:763) java.lang.Thread.run(Thread.java:636) Thread 28 (org.apache.hadoop.io.ObjectWritable Connection Culler): State: TIMED_WAITING Blocked count: 0 Waited count: 9848 Stack: java.lang.Thread.sleep(Native Method) org.apache.hadoop.ipc.Client$ConnectionCuller.run(Client.java:435) Thread 19 (org.apache.hadoop.io.ObjectWritable Connection Culler): State: TIMED_WAITING Blocked count: 6 Waited count: 9848 Stack: java.lang.Thread.sleep(Native Method) org.apache.hadoop.ipc.Client$ConnectionCuller.run(Client.java:435) Thread 18 (DestroyJavaVM): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 17 (regionserver/0:0:0:0:0:0:0:0:60020): State: TIMED_WAITING Blocked count: 2 Waited count: 6573 Stack: java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:72) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:417) java.lang.Thread.run(Thread.java:636) Thread 4 (Signal Dispatcher): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 3 (Finalizer): State: WAITING Blocked count: 4 Waited count: 128 Waiting on java.lang.ref.ReferenceQueue$Lock@9b962c9 Stack: java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177) Thread 2 (Reference Handler): State: WAITING Blocked count: 1 Waited count: 129 Waiting on java.lang.ref.Reference$Lock@4d34fbfd Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@4e727e0e, row => c08408b4-b68c-433c-ba3f-d46d3ba73288, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: discarded for being too old (1345396) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@518bc6a4, row => 8084b35c-9543-4d85-b23b-b615800c745b, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (1309902) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@2f9e62cf, row => c08408b4-b68c-433c-ba3f-d46d3ba73288, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: discarded for being too old (1275372) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@47691c75, row => 8084b35c-9543-4d85-b23b-b615800c745b, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (1239881) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@45cb69f3, row => c08408b4-b68c-433c-ba3f-d46d3ba73288, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: discarded for being too old (1205347) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@2d98f451, row => 8084b35c-9543-4d85-b23b-b615800c745b, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (1169857) 2008-07-31 16:03:09,479 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@14d0093a, row => c08408b4-b68c-433c-ba3f-d46d3ba73288, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: discarded for being too old (1135325) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@660dc982, row => 8084b35c-9543-4d85-b23b-b615800c745b, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (1099831) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (1048491) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@1328aa26, row => 8084b35c-9543-4d85-b23b-b615800c745b, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (1029809) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (987489) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@52baae9d, row => 8084b395-80b0-4a09-9a72-79b9c8d1f1ee, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (969806) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (926488) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@5e1defd9, row => 8084b395-80b0-4a09-9a72-79b9c8d1f1ee, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (899782) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (865488) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@7f6dbfd4, row => 8084b395-80b0-4a09-9a72-79b9c8d1f1ee, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (829757) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (804487) 2008-07-31 16:03:09,480 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@7e4d38a7, row => 8084b395-80b0-4a09-9a72-79b9c8d1f1ee, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (759729) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (743488) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@7fc29cb8, row => 8084b395-80b0-4a09-9a72-79b9c8d1f1ee, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (689703) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (682479) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@41bf358f, row => 8084b3b7-4033-49d8-894f-cab6b6ab5bbd, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (629701) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (621486) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (560486) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@70556101, row => 8084b3b7-4033-49d8-894f-cab6b6ab5bbd, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (559675) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (499485) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@7553c244, row => 8084b3b7-4033-49d8-894f-cab6b6ab5bbd, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (489648) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (438484) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@4df73a89, row => 8084b3b7-4033-49d8-894f-cab6b6ab5bbd, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (419621) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (377483) 2008-07-31 16:03:09,481 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@2224205c, row => 8084b3b7-4033-49d8-894f-cab6b6ab5bbd, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (349590) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (316483) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@71c88986, row => 8084b3fe-4d39-475c-89f3-146b33258a7f, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (289589) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (255482) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@24bcf6c6, row => 8084b3fe-4d39-475c-89f3-146b33258a7f, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (219565) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (194481) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@71ed5401, row => 8084b3fe-4d39-475c-89f3-146b33258a7f, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (149538) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (133481) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call batchUpdate([B@14264816, row => 8084b3fe-4d39-475c-89f3-146b33258a7f, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: discarded for being too old (79514) 2008-07-31 16:03:09,482 WARN org.apache.hadoop.ipc.Server: IPC Server handler 5 on 60020, call getProtocolVersion(org.apache.hadoop.hbase.ipc.HRegionInterface, 3) from 72.34.249.200:40821: discarded for being too old (72480) 2008-07-31 16:03:09,495 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 6 on 60020' 2008-07-31 16:03:09,521 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 2 on 60020' 2008-07-31 16:03:09,549 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 8 on 60020' 2008-07-31 16:03:09,576 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 1 on 60020' 2008-07-31 16:03:10,959 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/853828343/content/mapfiles/2345206955841469254 with 323686 entries, sequence id 5166782, data size 64.0m, file size 68.8m 2008-07-31 16:03:10,959 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 in 1899ms, sequence id=5166782, compaction requested=true 2008-07-31 16:03:10,959 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 9 on 60020' 2008-07-31 16:03:10,959 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 7 on 60020' 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 0 on 60020' 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 4 on 60020' 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [853828343/content/7150052340741078436, 853828343/content/899771229557496049, 853828343/content/2345206955841469254] 853828343/content 2008-07-31 16:03:10,960 INFO org.apache.hadoop.hbase.regionserver.HRegion: Unblocking updates for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 'IPC Server handler 3 on 60020' 2008-07-31 16:03:10,960 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call batchUpdate([B@284868c5, row => c08406fa-0945-464b-94b4-002746b7a67e, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: output error 2008-07-31 16:03:10,962 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call batchUpdate([B@488d3603, row => c08406fa-0945-464b-94b4-002746b7a67e, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: output error 2008-07-31 16:03:10,962 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 60020 caught: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342) at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:594) at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:654) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:917) 2008-07-31 16:03:10,962 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call batchUpdate([B@2ed5242b, row => c08406fa-0945-464b-94b4-002746b7a67e, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: output error 2008-07-31 16:03:10,962 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 60020 caught: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342) at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:594) at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:654) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:917) 2008-07-31 16:03:10,964 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call batchUpdate([B@2e94018b, row => c08406fa-0945-464b-94b4-002746b7a67e, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: output error 2008-07-31 16:03:10,966 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 60020 caught: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342) at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:594) at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:654) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:917) 2008-07-31 16:03:10,966 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 60020 caught: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342) at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:594) at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:654) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:917) 2008-07-31 16:03:10,967 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call batchUpdate([B@4a0d8a7b, row => c08406fa-0945-464b-94b4-002746b7a67e, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50815: output error 2008-07-31 16:03:10,967 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 60020 caught: java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342) at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:594) at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:654) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:917) 2008-07-31 16:03:10,970 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction size of 853828343/content: 275.3m, skipped 0, 0 2008-07-31 16:03:11,014 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 3 files into /hbase/items/compaction.dir/853828343/content/mapfiles/1129755312900160658 2008-07-31 16:03:11,489 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30009 2008-07-31 16:03:11,494 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543889125 2008-07-31 16:03:11,496 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545391494 2008-07-31 16:03:11,496 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Found 26 logs to remove using oldest outstanding seqnum of 5187925 from region [B@3541d281 2008-07-31 16:03:11,497 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543590597 whose highest sequence/edit id is 4435960 2008-07-31 16:03:11,499 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543592953 whose highest sequence/edit id is 4465968 2008-07-31 16:03:11,500 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543613786 whose highest sequence/edit id is 4495976 2008-07-31 16:03:11,501 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543616186 whose highest sequence/edit id is 4525984 2008-07-31 16:03:11,503 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543618569 whose highest sequence/edit id is 4555992 2008-07-31 16:03:11,504 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543662015 whose highest sequence/edit id is 4586000 2008-07-31 16:03:11,505 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543664715 whose highest sequence/edit id is 4616008 2008-07-31 16:03:11,506 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543667333 whose highest sequence/edit id is 4646016 2008-07-31 16:03:11,507 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543670001 whose highest sequence/edit id is 4676024 2008-07-31 16:03:11,508 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543675812 whose highest sequence/edit id is 4706032 2008-07-31 16:03:11,509 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543678246 whose highest sequence/edit id is 4737867 2008-07-31 16:03:11,510 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543680665 whose highest sequence/edit id is 4767875 2008-07-31 16:03:11,511 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543684696 whose highest sequence/edit id is 4797883 2008-07-31 16:03:11,512 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543698373 whose highest sequence/edit id is 4827891 2008-07-31 16:03:11,513 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543700859 whose highest sequence/edit id is 4857899 2008-07-31 16:03:11,514 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543703249 whose highest sequence/edit id is 4887907 2008-07-31 16:03:11,515 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543753629 whose highest sequence/edit id is 4917915 2008-07-31 16:03:11,516 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543756190 whose highest sequence/edit id is 4947923 2008-07-31 16:03:11,517 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543758514 whose highest sequence/edit id is 4977931 2008-07-31 16:03:11,518 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543815691 whose highest sequence/edit id is 5007939 2008-07-31 16:03:11,519 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543818397 whose highest sequence/edit id is 5037947 2008-07-31 16:03:11,520 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543821087 whose highest sequence/edit id is 5067955 2008-07-31 16:03:11,521 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543823856 whose highest sequence/edit id is 5097963 2008-07-31 16:03:11,522 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543879184 whose highest sequence/edit id is 5127971 2008-07-31 16:03:11,523 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543881887 whose highest sequence/edit id is 5166780 2008-07-31 16:03:11,524 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543884691 whose highest sequence/edit id is 5166781 2008-07-31 16:03:13,221 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:13,231 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217544549145 2008-07-31 16:03:13,234 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545393231 2008-07-31 16:03:15,775 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:15,899 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545391494 2008-07-31 16:03:15,900 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545395899 2008-07-31 16:03:17,542 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:17,589 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545393231 2008-07-31 16:03:17,590 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545397589 2008-07-31 16:03:19,032 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:19,037 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545395899 2008-07-31 16:03:19,039 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545399038 2008-07-31 16:03:20,441 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:20,449 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545397589 2008-07-31 16:03:20,451 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545400449 2008-07-31 16:03:21,849 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:03:21,854 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545399038 2008-07-31 16:03:21,856 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545401855 2008-07-31 16:03:25,800 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/853828343/content/mapfiles/1129755312900160658 to /hbase/items/853828343/content/mapfiles/5324551374993726109 2008-07-31 16:03:25,853 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 853828343/content store size is 275.0m 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [853828343/readby/2967318044136786382] 853828343/readby 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [853828343/savedby/8842526882507793138] 853828343/savedby 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [853828343/sentby/8867356411558358681] 853828343/sentby 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: false [853828343/receivedby/2219321021993316861] 853828343/receivedby 2008-07-31 16:03:25,856 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE THRESHOLD 2008-07-31 16:03:25,858 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 in 14sec 2008-07-31 16:03:25,860 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:25,862 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:25,862 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:25,862 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:25,862 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:25,862 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296. Current region memcache size 43.1m 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/853828343/content/mapfiles/8066674859873959521 with 220044 entries, sequence id 5386849, data size 43.1m, file size 46.3m 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 in 979ms, sequence id=5386849, compaction requested=false 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/uclusters 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/covisit 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/content 2008-07-31 16:03:26,841 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/clusters 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/readby 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/cfrecs 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/blooms 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/ranked 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/savedby 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/sentby 2008-07-31 16:03:26,842 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 853828343/receivedby 2008-07-31 16:03:26,842 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 2008-07-31 16:03:27,037 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861/477477829 2008-07-31 16:03:27,056 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/savedby/info/9064401644441351414.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,056 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/savedby, max sequence id 110420 2008-07-31 16:03:27,076 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/readby/info/7908427911470093623.853828343, isReference=true, sequence id=546972 2008-07-31 16:03:27,077 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/readby, max sequence id 546972 2008-07-31 16:03:27,089 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/receivedby/info/8534099389214484943.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,090 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/receivedby, max sequence id 110420 2008-07-31 16:03:27,123 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/sentby/info/4930689094817980988.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,123 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/sentby, max sequence id 110420 2008-07-31 16:03:27,136 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/content/info/2584284651423999831.853828343, isReference=true, sequence id=5166782 2008-07-31 16:03:27,142 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/content/info/8889272713903995706.853828343, isReference=true, sequence id=5386849 2008-07-31 16:03:27,142 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 477477829/content, max sequence id 5386849 2008-07-31 16:03:27,152 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 is 5386850 2008-07-31 16:03:27,153 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861/477477829 available 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/uclusters 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/covisit 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/content 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/clusters 2008-07-31 16:03:27,153 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/readby 2008-07-31 16:03:27,154 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/cfrecs 2008-07-31 16:03:27,154 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/blooms 2008-07-31 16:03:27,154 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/ranked 2008-07-31 16:03:27,154 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/savedby 2008-07-31 16:03:27,155 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/sentby 2008-07-31 16:03:27,155 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 477477829/receivedby 2008-07-31 16:03:27,155 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:27,155 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861/1788551205 2008-07-31 16:03:27,175 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/savedby/info/469153400718725822.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,175 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/savedby, max sequence id 110420 2008-07-31 16:03:27,187 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/readby/info/3374366515092571124.853828343, isReference=true, sequence id=546972 2008-07-31 16:03:27,188 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/readby, max sequence id 546972 2008-07-31 16:03:27,199 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/receivedby/info/4929847856753199534.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,200 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/receivedby, max sequence id 110420 2008-07-31 16:03:27,232 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/sentby/info/3984312064578370769.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:27,232 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/sentby, max sequence id 110420 2008-07-31 16:03:27,244 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/content/info/4151236935634093559.853828343, isReference=true, sequence id=5386849 2008-07-31 16:03:27,249 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/content/info/5300704574676956335.853828343, isReference=true, sequence id=5166782 2008-07-31 16:03:27,250 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 1788551205/content, max sequence id 5386849 2008-07-31 16:03:27,260 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 is 5386850 2008-07-31 16:03:27,261 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861/1788551205 available 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Compactions and cache flushes disabled for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates and scanners disabled for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more active scanners for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/uclusters 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/covisit 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/content 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/clusters 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/readby 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/cfrecs 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/blooms 2008-07-31 16:03:27,261 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/ranked 2008-07-31 16:03:27,262 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/savedby 2008-07-31 16:03:27,263 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/sentby 2008-07-31 16:03:27,263 DEBUG org.apache.hadoop.hbase.regionserver.HStore: closed 1788551205/receivedby 2008-07-31 16:03:27,263 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:27,265 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Cleaned up /hbase/items/853828343/splits true 2008-07-31 16:03:27,272 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: region split, META updated, and report to master all successful. Old region=REGION => {NAME => 'items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296', STARTKEY => '8001eb31-98bb-4087-bd8d-e4b42805addb', ENDKEY => '', ENCODED => 853828343, OFFLINE => true, SPLIT => true, TABLE => {{NAME => 'items', IS_ROOT => 'false', IS_META => 'false', FAMILIES => [{NAME => 'clusters', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'cfrecs', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'savedby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'readby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'receivedby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'ranked', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'blooms', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'covisit', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'uclusters', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'sentby', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}, {NAME => 'content', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '2', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}, new regions: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861, items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861. Split took 1sec 2008-07-31 16:03:28,962 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:28,962 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:28,962 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:28,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861/477477829 2008-07-31 16:03:28,976 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/savedby/info/9064401644441351414.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:28,976 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/savedby, max sequence id 110420 2008-07-31 16:03:28,992 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/readby/info/7908427911470093623.853828343, isReference=true, sequence id=546972 2008-07-31 16:03:28,993 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/readby, max sequence id 546972 2008-07-31 16:03:29,006 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/receivedby/info/8534099389214484943.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:29,007 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/receivedby, max sequence id 110420 2008-07-31 16:03:29,028 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/sentby/info/4930689094817980988.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:29,028 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 477477829/sentby, max sequence id 110420 2008-07-31 16:03:29,038 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/content/info/2584284651423999831.853828343, isReference=true, sequence id=5166782 2008-07-31 16:03:29,043 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/477477829/content/info/8889272713903995706.853828343, isReference=true, sequence id=5386849 2008-07-31 16:03:29,044 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 477477829/content, max sequence id 5386849 2008-07-31 16:03:29,054 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 is 5386850 2008-07-31 16:03:29,055 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861/477477829 available 2008-07-31 16:03:29,055 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:29,055 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:29,055 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861/1788551205 2008-07-31 16:03:29,056 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 2008-07-31 16:03:29,057 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [477477829/content/2584284651423999831-853828343/5324551374993726109/bottom, 477477829/content/8889272713903995706-853828343/8066674859873959521/bottom] 477477829/content 2008-07-31 16:03:29,113 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/savedby/info/469153400718725822.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:29,115 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 2 files into /hbase/items/compaction.dir/477477829/content/mapfiles/904322612929721585 2008-07-31 16:03:29,351 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/savedby, max sequence id 110420 2008-07-31 16:03:29,368 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/readby/info/3374366515092571124.853828343, isReference=true, sequence id=546972 2008-07-31 16:03:29,369 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/readby, max sequence id 546972 2008-07-31 16:03:29,384 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/receivedby/info/4929847856753199534.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:29,384 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/receivedby, max sequence id 110420 2008-07-31 16:03:29,406 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/sentby/info/3984312064578370769.853828343, isReference=true, sequence id=110420 2008-07-31 16:03:29,407 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore 1788551205/sentby, max sequence id 110420 2008-07-31 16:03:29,420 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/content/info/4151236935634093559.853828343, isReference=true, sequence id=5386849 2008-07-31 16:03:29,426 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /hbase/items/1788551205/content/info/5300704574676956335.853828343, isReference=true, sequence id=5166782 2008-07-31 16:03:29,427 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 2 file(s) in hstore 1788551205/content, max sequence id 5386849 2008-07-31 16:03:29,439 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 is 5386850 2008-07-31 16:03:29,440 INFO org.apache.hadoop.hbase.regionserver.HRegion: region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861/1788551205 available 2008-07-31 16:03:29,440 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region: items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:34,664 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/477477829/content/mapfiles/904322612929721585 to /hbase/items/477477829/content/mapfiles/1154421730165758639 2008-07-31 16:03:34,700 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 477477829/content store size is 137.1m 2008-07-31 16:03:34,700 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [477477829/readby/7908427911470093623-853828343/2967318044136786382/bottom] 477477829/readby 2008-07-31 16:03:34,707 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/477477829/readby/mapfiles/1487288950149951298 2008-07-31 16:03:34,785 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/477477829/readby/mapfiles/1487288950149951298 to /hbase/items/477477829/readby/mapfiles/7204538669449217655 2008-07-31 16:03:34,808 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 477477829/readby store size is 71.8k 2008-07-31 16:03:34,808 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [477477829/savedby/9064401644441351414-853828343/8842526882507793138/bottom] 477477829/savedby 2008-07-31 16:03:34,815 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/477477829/savedby/mapfiles/4796161626261067118 2008-07-31 16:03:34,970 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/477477829/savedby/mapfiles/4796161626261067118 to /hbase/items/477477829/savedby/mapfiles/5302262961470024589 2008-07-31 16:03:34,985 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 477477829/savedby store size is 4.8k 2008-07-31 16:03:34,985 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [477477829/sentby/4930689094817980988-853828343/8867356411558358681/bottom] 477477829/sentby 2008-07-31 16:03:34,995 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/477477829/sentby/mapfiles/9104956047227874436 2008-07-31 16:03:35,033 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/477477829/sentby/mapfiles/9104956047227874436 to /hbase/items/477477829/sentby/mapfiles/794869462939851808 2008-07-31 16:03:35,051 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 477477829/sentby store size is 895.0 2008-07-31 16:03:35,051 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [477477829/receivedby/8534099389214484943-853828343/2219321021993316861/bottom] 477477829/receivedby 2008-07-31 16:03:35,058 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/477477829/receivedby/mapfiles/8684413739977593014 2008-07-31 16:03:35,097 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/477477829/receivedby/mapfiles/8684413739977593014 to /hbase/items/477477829/receivedby/mapfiles/8718688867536230584 2008-07-31 16:03:35,113 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 477477829/receivedby store size is 915.0 2008-07-31 16:03:35,114 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217545405861 in 6sec 2008-07-31 16:03:35,116 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 2008-07-31 16:03:35,116 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [1788551205/content/5300704574676956335-853828343/5324551374993726109/top, 1788551205/content/4151236935634093559-853828343/8066674859873959521/top] 1788551205/content 2008-07-31 16:03:35,128 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 2 files into /hbase/items/compaction.dir/1788551205/content/mapfiles/12714374731098737 2008-07-31 16:03:45,875 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/1788551205/content/mapfiles/12714374731098737 to /hbase/items/1788551205/content/mapfiles/6568297087842014882 2008-07-31 16:03:45,912 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 1788551205/content store size is 184.3m 2008-07-31 16:03:45,912 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [1788551205/readby/3374366515092571124-853828343/2967318044136786382/top] 1788551205/readby 2008-07-31 16:03:45,918 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/1788551205/readby/mapfiles/8981248356972025846 2008-07-31 16:03:47,049 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/1788551205/readby/mapfiles/8981248356972025846 to /hbase/items/1788551205/readby/mapfiles/5075753606252548235 2008-07-31 16:03:47,069 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 1788551205/readby store size is 18.5m 2008-07-31 16:03:47,070 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [1788551205/savedby/469153400718725822-853828343/8842526882507793138/top] 1788551205/savedby 2008-07-31 16:03:47,075 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/1788551205/savedby/mapfiles/5164901112359051719 2008-07-31 16:03:47,166 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/1788551205/savedby/mapfiles/5164901112359051719 to /hbase/items/1788551205/savedby/mapfiles/4675616413857663854 2008-07-31 16:03:47,181 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 1788551205/savedby store size is 806.3k 2008-07-31 16:03:47,181 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [1788551205/sentby/3984312064578370769-853828343/8867356411558358681/top] 1788551205/sentby 2008-07-31 16:03:47,187 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/1788551205/sentby/mapfiles/2366646532423969497 2008-07-31 16:03:47,243 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/1788551205/sentby/mapfiles/2366646532423969497 to /hbase/items/1788551205/sentby/mapfiles/7750269442065960524 2008-07-31 16:03:47,258 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 1788551205/sentby store size is 319.0k 2008-07-31 16:03:47,258 INFO org.apache.hadoop.hbase.regionserver.HStore: REMOVE false, references: true [1788551205/receivedby/4929847856753199534-853828343/2219321021993316861/top] 1788551205/receivedby 2008-07-31 16:03:47,264 DEBUG org.apache.hadoop.hbase.regionserver.HStore: started compaction of 1 files into /hbase/items/compaction.dir/1788551205/receivedby/mapfiles/7763735740773054524 2008-07-31 16:03:47,320 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /hbase/items/compaction.dir/1788551205/receivedby/mapfiles/7763735740773054524 to /hbase/items/1788551205/receivedby/mapfiles/5362746827190670023 2008-07-31 16:03:47,334 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 1788551205/receivedby store size is 328.5k 2008-07-31 16:03:47,335 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 in 12sec 2008-07-31 16:04:00,666 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 60020, call batchUpdate([B@480383c5, row => c015f477-f94a-426b-8919-39ef835f9610, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:40821: error: org.apache.hadoop.hbase.NotServingRegionException: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 org.apache.hadoop.hbase.NotServingRegionException: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:1436) at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1147) at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) 2008-07-31 16:04:12,516 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30009 2008-07-31 16:04:12,524 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545400449 2008-07-31 16:04:12,525 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545452524 2008-07-31 16:04:12,525 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Found 7 logs to remove using oldest outstanding seqnum of 5386850 from region [B@43b4e65a 2008-07-31 16:04:12,525 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217543889125 whose highest sequence/edit id is 5196790 2008-07-31 16:04:12,530 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217544549145 whose highest sequence/edit id is 5226798 2008-07-31 16:04:12,531 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545391494 whose highest sequence/edit id is 5256806 2008-07-31 16:04:12,532 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545393231 whose highest sequence/edit id is 5286814 2008-07-31 16:04:12,534 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545395899 whose highest sequence/edit id is 5316822 2008-07-31 16:04:12,536 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545397589 whose highest sequence/edit id is 5346830 2008-07-31 16:04:12,537 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545399038 whose highest sequence/edit id is 5376838 2008-07-31 16:04:15,201 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:15,246 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545401855 2008-07-31 16:04:15,247 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545455246 2008-07-31 16:04:17,837 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:17,844 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545452524 2008-07-31 16:04:17,845 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545457844 2008-07-31 16:04:22,182 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 60020, call batchUpdate([B@15dd5f6a, row => 809ac928-c151-4778-8b2a-d2557f770058, {column => content:title, value => '...', column => content:content, value => '...', column => content:sourceid, value => '...', column => content:url, value => '...', column => content:addstamp, value => '...', column => content:xml, value => '...', column => content:feedstamp, value => '...', column => content:author, value => '...', column => content:preview, value => '...', column => content:thumb, value => '...', column => content:attachments, value => '...'}) from 72.34.249.200:50821: error: org.apache.hadoop.hbase.NotServingRegionException: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 org.apache.hadoop.hbase.NotServingRegionException: items,8001eb31-98bb-4087-bd8d-e4b42805addb,1217543451296 at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:1436) at org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1147) at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:473) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896) 2008-07-31 16:04:32,225 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:32,231 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545455246 2008-07-31 16:04:32,233 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545472231 2008-07-31 16:04:34,614 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:34,621 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545457844 2008-07-31 16:04:34,623 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545474621 2008-07-31 16:04:37,132 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:37,139 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545472231 2008-07-31 16:04:37,141 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545477139 2008-07-31 16:04:39,749 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:39,757 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545474621 2008-07-31 16:04:39,759 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545479757 2008-07-31 16:04:55,459 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:55,466 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545477139 2008-07-31 16:04:55,467 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545495466 2008-07-31 16:04:58,491 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:04:58,501 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545479757 2008-07-31 16:04:58,502 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545498501 2008-07-31 16:05:00,850 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:00,859 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545495466 2008-07-31 16:05:00,860 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545500859 2008-07-31 16:05:03,184 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:03,193 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545498501 2008-07-31 16:05:03,195 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545503193 2008-07-31 16:05:03,701 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memcache flush for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861. Current region memcache size 64.0m 2008-07-31 16:05:06,092 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Added /hbase/items/1788551205/content/mapfiles/2878248648518485646 with 326271 entries, sequence id 5713121, data size 64.0m, file size 68.8m 2008-07-31 16:05:06,092 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memcache flush for region items,808496d3-c077-41e0-9ef5-f33d45227ff3,1217545405861 in 2391ms, sequence id=5713121, compaction requested=false 2008-07-31 16:05:48,812 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30009 2008-07-31 16:05:48,820 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545500859 2008-07-31 16:05:48,821 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545548820 2008-07-31 16:05:48,821 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Found 11 logs to remove using oldest outstanding seqnum of 5716851 from region [B@43b4e65a 2008-07-31 16:05:48,822 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545400449 whose highest sequence/edit id is 5406847 2008-07-31 16:05:48,823 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545401855 whose highest sequence/edit id is 5436855 2008-07-31 16:05:48,825 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545452524 whose highest sequence/edit id is 5466863 2008-07-31 16:05:48,827 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545455246 whose highest sequence/edit id is 5496871 2008-07-31 16:05:48,829 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545457844 whose highest sequence/edit id is 5526879 2008-07-31 16:05:48,831 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545472231 whose highest sequence/edit id is 5556887 2008-07-31 16:05:48,833 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545474621 whose highest sequence/edit id is 5586895 2008-07-31 16:05:48,835 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545477139 whose highest sequence/edit id is 5616903 2008-07-31 16:05:48,836 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545479757 whose highest sequence/edit id is 5646911 2008-07-31 16:05:48,837 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545495466 whose highest sequence/edit id is 5676919 2008-07-31 16:05:48,839 INFO org.apache.hadoop.hbase.regionserver.HLog: removing old log file /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545498501 whose highest sequence/edit id is 5706927 2008-07-31 16:05:50,429 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:50,438 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545503193 2008-07-31 16:05:50,439 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545550438 2008-07-31 16:05:51,823 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:51,828 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545548820 2008-07-31 16:05:51,830 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545551828 2008-07-31 16:05:53,148 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:53,154 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545550438 2008-07-31 16:05:53,156 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545553155 2008-07-31 16:05:54,459 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:54,467 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545551828 2008-07-31 16:05:54,468 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545554467 2008-07-31 16:05:55,793 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of entries: 30008 2008-07-31 16:05:55,799 DEBUG org.apache.hadoop.hbase.regionserver.HLog: Closing current log writer /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545553155 2008-07-31 16:05:55,801 INFO org.apache.hadoop.hbase.regionserver.HLog: New log writer created at /hbase/log_72.34.249.212_1217535541159_60020/hlog.dat.1217545555800