2009-08-26 05:25:06,942 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1689017977/copy/5061481606567458045; store size is 128.6m 2009-08-26 05:25:06,943 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE053526169\x80m\xDF\x7F\xFF\xFF\xFE,1251256941743 in 13sec 2009-08-26 05:25:06,945 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE176821519\xFA\xFBO\x7F\xFF\xFF\xF2,1251256805334 2009-08-26 05:25:06,948 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of batchid: 32.3m; Skipped 1 file(s), size: 22442202 2009-08-26 05:25:06,948 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/1182953014, seqid=27236 2009-08-26 05:25:09,372 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1182953014/batchid/4661090226714398718; store size is 32.3m 2009-08-26 05:25:09,373 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of copy: 217.6m; Skipped 1 file(s), size: 150963224 2009-08-26 05:25:09,373 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/1182953014, seqid=27236 2009-08-26 05:25:14,193 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1182953014/copy/5755406100294047333; store size is 217.6m 2009-08-26 05:25:14,195 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE176821519\xFA\xFBO\x7F\xFF\xFF\xF2,1251256805334 in 7sec 2009-08-26 05:25:14,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE022819586\x90\xE8\xCF\x7F\xFF\xFF\xFB,1251256951041 2009-08-26 05:25:14,199 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/950349958, seqid=27221 2009-08-26 05:25:24,419 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/950349958/batchid/3960029655590591170; store size is 20.9m 2009-08-26 05:25:24,421 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/950349958, seqid=27221 2009-08-26 05:25:36,856 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/950349958/copy/6229849783124561591; store size is 140.8m 2009-08-26 05:25:36,857 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE022819586\x90\xE8\xCF\x7F\xFF\xFF\xFB,1251256951041 in 22sec 2009-08-26 05:25:36,858 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE023309479\x80m\xDF\x7F\xFF\xFF\xFC,1251256951041 2009-08-26 05:25:36,863 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1994394616, seqid=27222 2009-08-26 05:25:38,048 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985 2009-08-26 05:25:38,048 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:38,048 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985 2009-08-26 05:25:38,048 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985, encoded=1897448391 2009-08-26 05:25:38,116 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/batchid/1458657344531703261.2079620806, isReference=true, sequence id=27787, length=25950750, majorCompaction=false 2009-08-26 05:25:38,132 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/batchid/173068843293839889.2079620806, isReference=true, sequence id=27800, length=16496392, majorCompaction=false 2009-08-26 05:25:38,177 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/batchid/4838943575328456877.2079620806, isReference=true, sequence id=27803, length=4292080, majorCompaction=false 2009-08-26 05:25:38,219 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/batchid/6009173667580077935.2079620806, isReference=true, sequence id=27801, length=5498666, majorCompaction=false 2009-08-26 05:25:38,292 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/copy/2829443971404369943.2079620806, isReference=true, sequence id=27803, length=28870128, majorCompaction=false 2009-08-26 05:25:38,347 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/copy/3219958223115296648.2079620806, isReference=true, sequence id=27800, length=110966526, majorCompaction=false 2009-08-26 05:25:38,373 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/copy/8489495165680839125.2079620806, isReference=true, sequence id=27801, length=36986860, majorCompaction=false 2009-08-26 05:25:38,458 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1897448391/copy/8497076366369261907.2079620806, isReference=true, sequence id=27787, length=174564398, majorCompaction=false 2009-08-26 05:25:38,467 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985/1897448391 available; sequence id is 27804 2009-08-26 05:25:38,468 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985/1897448391 because: Region has references on open 2009-08-26 05:25:38,468 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:38,468 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985, encoded=1910188893 2009-08-26 05:25:38,529 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/batchid/1458657344531703261.2079620806, isReference=true, sequence id=27788, length=25950750, majorCompaction=false 2009-08-26 05:25:39,904 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/batchid/173068843293839889.2079620806, isReference=true, sequence id=27801, length=16496392, majorCompaction=false 2009-08-26 05:25:39,912 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/batchid/4838943575328456877.2079620806, isReference=true, sequence id=27804, length=4292080, majorCompaction=false 2009-08-26 05:25:39,924 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/batchid/6009173667580077935.2079620806, isReference=true, sequence id=27802, length=5498666, majorCompaction=false 2009-08-26 05:25:39,948 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/copy/2829443971404369943.2079620806, isReference=true, sequence id=27804, length=28870128, majorCompaction=false 2009-08-26 05:25:39,974 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/copy/3219958223115296648.2079620806, isReference=true, sequence id=27801, length=110966526, majorCompaction=false 2009-08-26 05:25:39,987 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/copy/8489495165680839125.2079620806, isReference=true, sequence id=27802, length=36986860, majorCompaction=false 2009-08-26 05:25:40,002 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1910188893/copy/8497076366369261907.2079620806, isReference=true, sequence id=27788, length=174564398, majorCompaction=false 2009-08-26 05:25:40,004 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 available; sequence id is 27805 2009-08-26 05:25:40,004 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 because: Region has references on open 2009-08-26 05:25:41,099 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1994394616/batchid/2341214477092201020; store size is 20.9m 2009-08-26 05:25:41,101 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 3 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1994394616, seqid=27222 2009-08-26 05:25:41,348 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=5.013962MB (5257520), Free=606.06104MB (635501072), Max=611.075MB (640758592), Counts: Blocks=0, Access=2109974, Hit=0, Miss=2109974, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN 2009-08-26 05:25:48,439 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:48,440 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985. Current region memstore size 64.0m 2009-08-26 05:25:50,729 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/batchid/8588579144214004158, entries=99882, sequenceid=27806, memsize=16.8m, filesize=5.2m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:53,411 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/copy/6533526894037100300, entries=99882, sequenceid=27806, memsize=47.2m, filesize=35.3m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:53,412 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 in 4972ms, sequence id=27806, compaction requested=true 2009-08-26 05:25:53,412 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:25:53,780 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:53,780 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985. Current region memstore size 64.0m 2009-08-26 05:25:54,440 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/batchid/6455693681466624897, entries=99881, sequenceid=27807, memsize=16.8m, filesize=5.2m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:55,147 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/copy/8721249159652966039, entries=99881, sequenceid=27807, memsize=47.2m, filesize=35.3m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:55,148 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 in 1368ms, sequence id=27807, compaction requested=true 2009-08-26 05:25:55,148 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:25:58,303 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:58,304 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985. Current region memstore size 64.0m 2009-08-26 05:25:58,663 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/batchid/7034133142842407442, entries=99869, sequenceid=27808, memsize=16.8m, filesize=5.2m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:58,817 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1994394616/copy/8922319162750829784; store size is 140.8m 2009-08-26 05:25:58,867 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE023309479\x80m\xDF\x7F\xFF\xFF\xFC,1251256951041 in 22sec 2009-08-26 05:25:58,877 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE177135908\x03l_\x7F\xFF\xFF\xFE,1251256805334 2009-08-26 05:25:58,893 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of batchid: 34.1m; Skipped 1 file(s), size: 22450299 2009-08-26 05:25:58,893 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/951254408, seqid=27240 2009-08-26 05:25:59,808 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/copy/582542798425683111, entries=99869, sequenceid=27808, memsize=47.2m, filesize=35.3m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:25:59,808 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 in 1504ms, sequence id=27808, compaction requested=true 2009-08-26 05:25:59,808 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:25:59,982 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/951254408/batchid/8718046008210330916; store size is 34.1m 2009-08-26 05:25:59,990 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of copy: 229.3m; Skipped 1 file(s), size: 151017307 2009-08-26 05:25:59,990 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/951254408, seqid=27240 2009-08-26 05:26:02,110 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:26:02,110 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985. Current region memstore size 64.0m 2009-08-26 05:26:02,409 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/batchid/5119067998952156892, entries=99879, sequenceid=27809, memsize=16.8m, filesize=5.2m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:26:03,973 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/copy/222836993981193833, entries=99879, sequenceid=27809, memsize=47.2m, filesize=35.3m to domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:26:03,973 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 in 1863ms, sequence id=27809, compaction requested=true 2009-08-26 05:26:03,973 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985/1910188893 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:26:07,873 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/951254408/copy/2840797751242702097; store size is 229.3m 2009-08-26 05:26:07,876 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE177135908\x03l_\x7F\xFF\xFF\xFE,1251256805334 in 8sec 2009-08-26 05:26:07,888 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE181121109t\x90_\x7F\xFF\xFF\xFE,1251257023139 2009-08-26 05:26:07,893 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1602491782, seqid=27245 2009-08-26 05:26:07,898 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_7352946847213562224_926536 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:11,107 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_7352946847213562224_926536 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:14,111 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/batchid/5233728315379016023 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1770) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1586) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1713) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(BoundedRangeFileInputStream.java:99) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:979) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:936) at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.seekTo(HFile.java:1249) at org.apache.hadoop.hbase.io.HalfHFileReader$1.seekTo(HalfHFileReader.java:174) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:98) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:68) at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.(MinorCompactingStoreScanner.java:47) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:893) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:738) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:779) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:104) 2009-08-26 05:26:14,122 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_29003184764801127_927043 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:14,386 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:26:14,387 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216. Current region memstore size 64.0m 2009-08-26 05:26:14,693 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/7755192165297730353, entries=99941, sequenceid=27810, memsize=16.8m, filesize=5.2m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:26:17,137 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_29003184764801127_927043 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:17,279 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/101333027592972130, entries=99941, sequenceid=27810, memsize=47.3m, filesize=35.3m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:26:17,280 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 2892ms, sequence id=27810, compaction requested=true 2009-08-26 05:26:17,280 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216/336915850 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:26:20,144 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/batchid/6624963520417627461 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1770) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1586) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1713) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(BoundedRangeFileInputStream.java:99) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:979) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:936) at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.seekTo(HFile.java:1249) at org.apache.hadoop.hbase.io.HalfHFileReader$1.seekTo(HalfHFileReader.java:174) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:98) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:68) at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.(MinorCompactingStoreScanner.java:47) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:893) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:738) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:779) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:104) 2009-08-26 05:26:24,123 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1602491782/batchid/1907549853541520926; store size is 5.2m 2009-08-26 05:26:24,133 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1602491782, seqid=27245 2009-08-26 05:26:27,901 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_1701447019709194276_927070 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:30,919 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_1701447019709194276_927070 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:33,923 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/copy/6957658033169988104 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1770) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1586) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1713) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(BoundedRangeFileInputStream.java:99) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:979) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:936) at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.seekTo(HFile.java:1249) at org.apache.hadoop.hbase.io.HalfHFileReader$1.seekTo(HalfHFileReader.java:174) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:98) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:68) at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.(MinorCompactingStoreScanner.java:47) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:893) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:738) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:779) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:104) 2009-08-26 05:26:40,363 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:26:40,364 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216. Current region memstore size 64.0m 2009-08-26 05:26:40,722 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/batchid/12275338848430579, entries=99880, sequenceid=27811, memsize=16.8m, filesize=5.2m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:26:41,348 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=5.013962MB (5257520), Free=606.06104MB (635501072), Max=611.075MB (640758592), Counts: Blocks=0, Access=2114917, Hit=0, Miss=2114917, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN 2009-08-26 05:26:42,159 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/copy/3564593685326244236, entries=99880, sequenceid=27811, memsize=47.2m, filesize=35.3m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:26:42,159 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 in 1795ms, sequence id=27811, compaction requested=true 2009-08-26 05:26:42,159 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216/1750440440 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:26:43,942 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_-3210782409023123156_926549 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:46,949 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block blk_-3210782409023123156_926549 from any node: java.io.IOException: No live nodes contain current block 2009-08-26 05:26:49,954 WARN org.apache.hadoop.hdfs.DFSClient: DFS Read: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/copy/5493441849584602083 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1770) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1586) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1713) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(BoundedRangeFileInputStream.java:99) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:979) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:936) at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1118) at org.apache.hadoop.hbase.io.HalfHFileReader$1.next(HalfHFileReader.java:125) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:76) at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.next(MinorCompactingStoreScanner.java:123) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:895) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:738) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:779) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:104) 2009-08-26 05:26:49,967 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region domirama,0TE181121109t\x90_\x7F\xFF\xFF\xFE,1251257023139 java.lang.RuntimeException: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/copy/5493441849584602083 at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:61) at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:76) at org.apache.hadoop.hbase.regionserver.MinorCompactingStoreScanner.next(MinorCompactingStoreScanner.java:123) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:895) at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:738) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:779) at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:104) Caused by: java.io.IOException: Cannot open filename /hdfs/hadoop/hbase/domirama/377086364/copy/5493441849584602083 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1770) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1586) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1713) at java.io.DataInputStream.read(DataInputStream.java:132) at org.apache.hadoop.hbase.io.hfile.BoundedRangeFileInputStream.read(BoundedRangeFileInputStream.java:99) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:979) at org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:936) at org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1118) at org.apache.hadoop.hbase.io.HalfHFileReader$1.next(HalfHFileReader.java:125) at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:58) ... 7 more 2009-08-26 05:26:49,969 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:26:49,972 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/336915850, seqid=27810 2009-08-26 05:26:52,678 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/2381134036524383356; store size is 31.1m 2009-08-26 05:26:52,679 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/336915850, seqid=27810 2009-08-26 05:27:00,028 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/7465667704840804529; store size is 209.3m 2009-08-26 05:27:00,031 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 10sec 2009-08-26 05:27:00,036 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:00,042 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1750440440, seqid=27811 2009-08-26 05:27:03,590 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/batchid/4120047183385519720; store size is 35.9m 2009-08-26 05:27:03,591 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1750440440, seqid=27811 2009-08-26 05:27:08,650 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:08,653 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216. Current region memstore size 64.0m 2009-08-26 05:27:09,359 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/4266346162513766926, entries=99872, sequenceid=27812, memsize=16.8m, filesize=5.2m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:09,447 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/copy/7714146800890977224; store size is 241.2m 2009-08-26 05:27:09,449 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 in 9sec 2009-08-26 05:27:09,452 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985 2009-08-26 05:27:09,454 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1897448391, seqid=27803 2009-08-26 05:27:11,754 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/5745043288219017664, entries=99872, sequenceid=27812, memsize=47.2m, filesize=35.3m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:11,754 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 3101ms, sequence id=27812, compaction requested=false 2009-08-26 05:27:11,879 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1897448391/batchid/975246829006005307; store size is 34.2m 2009-08-26 05:27:11,880 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 4 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1897448391, seqid=27803 2009-08-26 05:27:16,404 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:16,404 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216. Current region memstore size 64.0m 2009-08-26 05:27:16,718 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/4279385320069740553, entries=99876, sequenceid=27813, memsize=16.8m, filesize=5.2m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:18,366 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/2256667572774742306, entries=99876, sequenceid=27813, memsize=47.2m, filesize=35.3m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:18,367 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 1963ms, sequence id=27813, compaction requested=true 2009-08-26 05:27:18,367 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216/336915850 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:27:18,784 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:18,785 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216. Current region memstore size 64.1m 2009-08-26 05:27:19,166 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/batchid/3131531839738679518, entries=99968, sequenceid=27814, memsize=16.8m, filesize=5.2m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:20,479 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/copy/3991230435437889941, entries=99968, sequenceid=27814, memsize=47.3m, filesize=35.3m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:20,479 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.1m for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 in 1694ms, sequence id=27814, compaction requested=false 2009-08-26 05:27:29,261 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1897448391/copy/5799806358371138720; store size is 230.1m 2009-08-26 05:27:29,262 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE197735959\xBEd_\x7F\xFF\xFF\xFA,1251257120985 in 19sec 2009-08-26 05:27:29,264 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 2009-08-26 05:27:29,267 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 8 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1910188893, seqid=27809 2009-08-26 05:27:33,088 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/batchid/1190456790859219463; store size is 36.6m 2009-08-26 05:27:33,090 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 8 file(s), hasReferences=true, into /hdfs/hadoop/hbase/domirama/compaction.dir/1910188893, seqid=27809 2009-08-26 05:27:33,780 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Flush requested on domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:33,780 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216. Current region memstore size 64.0m 2009-08-26 05:27:34,177 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/batchid/6043248365203937181, entries=99870, sequenceid=27815, memsize=16.8m, filesize=5.2m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:35,204 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/copy/2673487497838997697, entries=99870, sequenceid=27815, memsize=47.2m, filesize=35.3m to domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:35,204 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~64.0m for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 in 1424ms, sequence id=27815, compaction requested=true 2009-08-26 05:27:35,204 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216/1750440440 because: regionserver/10.154.99.182:60020.cacheFlusher 2009-08-26 05:27:37,559 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: domirama,0TE21103966:-\x14O\x7F\xFF\xFF\xFD,1251257249295 2009-08-26 05:27:37,559 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN: domirama,0TE208317708\xC3\xD9O\x7F\xFF\xFF\xFE,1251257249295 2009-08-26 05:27:37,559 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: domirama,0TE21103966:-\x14O\x7F\xFF\xFF\xFD,1251257249295 2009-08-26 05:27:37,560 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE21103966:-\x14O\x7F\xFF\xFF\xFD,1251257249295, encoded=372082582 2009-08-26 05:27:37,603 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/batchid/4076892986644041772.1599484560, isReference=true, sequence id=27832, length=5499491, majorCompaction=false 2009-08-26 05:27:37,615 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/batchid/4775600028707252969.1599484560, isReference=true, sequence id=27834, length=6167586, majorCompaction=false 2009-08-26 05:27:37,625 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/batchid/8776525789583442044.1599484560, isReference=true, sequence id=27829, length=51883700, majorCompaction=false 2009-08-26 05:27:37,642 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/copy/1868655168351664666.1599484560, isReference=true, sequence id=27832, length=36992040, majorCompaction=false 2009-08-26 05:27:37,666 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/copy/4891432893609272617.1599484560, isReference=true, sequence id=27829, length=349011149, majorCompaction=false 2009-08-26 05:27:37,675 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/372082582/copy/9142869369810017405.1599484560, isReference=true, sequence id=27834, length=41487014, majorCompaction=false 2009-08-26 05:27:37,684 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE21103966:-\x14O\x7F\xFF\xFF\xFD,1251257249295/372082582 available; sequence id is 27835 2009-08-26 05:27:37,684 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE21103966:-\x14O\x7F\xFF\xFF\xFD,1251257249295/372082582 because: Region has references on open 2009-08-26 05:27:37,684 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: domirama,0TE208317708\xC3\xD9O\x7F\xFF\xFF\xFE,1251257249295 2009-08-26 05:27:37,685 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE208317708\xC3\xD9O\x7F\xFF\xFF\xFE,1251257249295, encoded=1888728922 2009-08-26 05:27:37,709 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/batchid/4076892986644041772.1599484560, isReference=true, sequence id=27831, length=5499491, majorCompaction=false 2009-08-26 05:27:37,746 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/batchid/4775600028707252969.1599484560, isReference=true, sequence id=27833, length=6167586, majorCompaction=false 2009-08-26 05:27:37,763 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/batchid/8776525789583442044.1599484560, isReference=true, sequence id=27828, length=51883700, majorCompaction=false 2009-08-26 05:27:37,791 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/copy/1868655168351664666.1599484560, isReference=true, sequence id=27831, length=36992040, majorCompaction=false 2009-08-26 05:27:37,817 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/copy/4891432893609272617.1599484560, isReference=true, sequence id=27828, length=349011149, majorCompaction=false 2009-08-26 05:27:37,829 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1888728922/copy/9142869369810017405.1599484560, isReference=true, sequence id=27833, length=41487014, majorCompaction=false 2009-08-26 05:27:37,838 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE208317708\xC3\xD9O\x7F\xFF\xFF\xFE,1251257249295/1888728922 available; sequence id is 27834 2009-08-26 05:27:37,838 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested for region domirama,0TE208317708\xC3\xD9O\x7F\xFF\xFF\xFE,1251257249295/1888728922 because: Region has references on open 2009-08-26 05:27:41,348 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes: Total=5.013962MB (5257520), Free=606.06104MB (635501072), Max=611.075MB (640758592), Counts: Blocks=0, Access=2132534, Hit=0, Miss=2132534, Evictions=0, Evicted=0, Ratios: Hit Ratio=0.0%, Miss Ratio=100.0%, Evicted/Run=NaN 2009-08-26 05:27:41,963 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1910188893/copy/6257637602748362741; store size is 246.1m 2009-08-26 05:27:41,964 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE198937707\x5B\xFD\xDF\x7F\xFF\xFF\xFE,1251257120985 in 12sec 2009-08-26 05:27:41,966 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:41,968 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of batchid: 41.6m; Skipped 1 file(s), size: 32632934 2009-08-26 05:27:41,968 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/336915850, seqid=27813 2009-08-26 05:27:42,658 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/7927697219431402671; store size is 41.6m 2009-08-26 05:27:42,659 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of copy: 279.9m; Skipped 1 file(s), size: 219514264 2009-08-26 05:27:42,659 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/336915850, seqid=27813 2009-08-26 05:27:44,631 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of copy; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/4849222272364399366; store size is 279.9m 2009-08-26 05:27:44,633 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 2sec 2009-08-26 05:27:44,739 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:44,745 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216: compactions & flushes disabled 2009-08-26 05:27:44,745 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:44,745 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:44,745 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216. Current region memstore size 18.4m 2009-08-26 05:27:44,862 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/batchid/540399168740175, entries=28772, sequenceid=27836, memsize=4.8m, filesize=1.5m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:45,151 DEBUG org.apache.hadoop.hbase.regionserver.Store: Added hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/336915850/copy/3087858500720548240, entries=28772, sequenceid=27836, memsize=13.6m, filesize=10.2m to domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:45,151 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Finished memstore flush of ~18.4m for region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 in 406ms, sequence id=27836, compaction requested=true 2009-08-26 05:27:45,151 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed batchid 2009-08-26 05:27:45,151 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed copy 2009-08-26 05:27:45,151 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216 2009-08-26 05:27:47,410 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742, encoded=7879186 2009-08-26 05:27:47,426 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/batchid/2381134036524383356.336915850, isReference=true, sequence id=27810, length=32632934, majorCompaction=false 2009-08-26 05:27:47,432 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/batchid/540399168740175.336915850, isReference=true, sequence id=27836, length=1584433, majorCompaction=false 2009-08-26 05:27:47,436 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/batchid/7927697219431402671.336915850, isReference=true, sequence id=27813, length=10997694, majorCompaction=false 2009-08-26 05:27:47,441 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/copy/3087858500720548240.336915850, isReference=true, sequence id=27836, length=10656140, majorCompaction=false 2009-08-26 05:27:47,447 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/copy/4849222272364399366.336915850, isReference=true, sequence id=27813, length=73977803, majorCompaction=false 2009-08-26 05:27:47,458 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/7879186/copy/7465667704840804529.336915850, isReference=true, sequence id=27810, length=219514264, majorCompaction=false 2009-08-26 05:27:47,471 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742/7879186 available; sequence id is 27837 2009-08-26 05:27:47,471 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742: compactions & flushes disabled 2009-08-26 05:27:47,471 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742 2009-08-26 05:27:47,471 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742 2009-08-26 05:27:47,471 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed batchid 2009-08-26 05:27:47,471 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed copy 2009-08-26 05:27:47,472 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742 2009-08-26 05:27:47,472 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742, encoded=1980366830 2009-08-26 05:27:47,499 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/batchid/2381134036524383356.336915850, isReference=true, sequence id=27811, length=32632934, majorCompaction=false 2009-08-26 05:27:47,510 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/batchid/540399168740175.336915850, isReference=true, sequence id=27837, length=1584433, majorCompaction=false 2009-08-26 05:27:47,514 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/batchid/7927697219431402671.336915850, isReference=true, sequence id=27814, length=10997694, majorCompaction=false 2009-08-26 05:27:47,519 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/copy/3087858500720548240.336915850, isReference=true, sequence id=27837, length=10656140, majorCompaction=false 2009-08-26 05:27:47,526 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/copy/4849222272364399366.336915850, isReference=true, sequence id=27814, length=73977803, majorCompaction=false 2009-08-26 05:27:47,544 DEBUG org.apache.hadoop.hbase.regionserver.Store: loaded /hdfs/hadoop/hbase/domirama/1980366830/copy/7465667704840804529.336915850, isReference=true, sequence id=27811, length=219514264, majorCompaction=false 2009-08-26 05:27:47,545 INFO org.apache.hadoop.hbase.regionserver.HRegion: region domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742/1980366830 available; sequence id is 27838 2009-08-26 05:27:47,545 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742: compactions & flushes disabled 2009-08-26 05:27:47,545 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Updates disabled for region, no outstanding scanners on domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742 2009-08-26 05:27:47,546 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: No more row locks outstanding on region domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742 2009-08-26 05:27:47,546 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed batchid 2009-08-26 05:27:47,546 DEBUG org.apache.hadoop.hbase.regionserver.Store: closed copy 2009-08-26 05:27:47,546 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742 2009-08-26 05:27:47,547 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Cleaned up /hdfs/hadoop/hbase/domirama/336915850/splits true 2009-08-26 05:27:47,556 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: region split, META updated, and report to master all successful. Old region=REGION => {NAME => 'domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257028216', STARTKEY => '0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE', ENDKEY => '0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD', ENCODED => 336915850, OFFLINE => true, SPLIT => true, TABLE => {{NAME => 'domirama', FAMILIES => [{NAME => 'batchid', VERSIONS => '3', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}, {NAME => 'copy', VERSIONS => '1', COMPRESSION => 'NONE', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}, new regions: domirama,0TE190717939\x15\xA4_\x7F\xFF\xFF\xFE,1251257264742, domirama,0TE19101402:8\xF1\xCF\x7F\xFF\xFF\xFC,1251257264742. Split took 2sec 2009-08-26 05:27:47,556 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region domirama,0TE191628736\xEB\xE0O\x7F\xFF\xFF\xFD,1251257028216 2009-08-26 05:27:47,560 DEBUG org.apache.hadoop.hbase.regionserver.Store: Compaction size of batchid: 46.4m; Skipped 1 file(s), size: 37605681 2009-08-26 05:27:47,560 DEBUG org.apache.hadoop.hbase.regionserver.Store: Started compaction of 2 file(s) into /hdfs/hadoop/hbase/domirama/compaction.dir/1750440440, seqid=27815 2009-08-26 05:27:48,304 DEBUG org.apache.hadoop.hbase.regionserver.Store: Completed compaction of batchid; new storefile is hdfs://hadooptux-00.gicm.net:8020/hdfs/hadoop/hbase/domirama/1750440440/batchid/1058467762601197393; store size is 46.4m