Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Cannot Reproduce
-
0.94.1
-
None
-
None
-
Ubuntu Server 10.04, Hadoop 1.0.3
Description
The 'splitCount' in this line is zero in some scenario, then throw ArithmeticException: / by zero, and the '_balancedSplit' file was not deleted:
LOG.debug("Avg Time / Split = "
+ org.apache.hadoop.util.StringUtils.formatTime(tDiff / splitCount));
Steps to reproduce:
shell> create 'test2', 'i' shell> for i in 'a'..'z' do for j in 'a'..'z' do put 'test2', "#{i}#{j}", "i:#{j}", "#{j}" end end
$ bin/hbase org.apache.hadoop.hbase.util.RegionSplitter -r -o 2 test2 HexStringSplit 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:host.name=dev-vm0 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_29 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/jdk1.6.0_29/jre 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/opt/hbase/bin/../conf:/usr/lib/jvm/default-java/lib/tools.jar:/opt/hbase/bin/..:/opt/hbase/bin/../hbase-0.94.1.jar:/opt/hbase/bin/../hbase-0.94.1-tests.jar:/opt/hbase/bin/../lib/activation-1.1.jar:/opt/hbase/bin/../lib/asm-3.1.jar:/opt/hbase/bin/../lib/avro-1.5.3.jar:/opt/hbase/bin/../lib/avro-ipc-1.5.3.jar:/opt/hbase/bin/../lib/commons-beanutils-1.7.0.jar:/opt/hbase/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/hbase/bin/../lib/commons-cli-1.2.jar:/opt/hbase/bin/../lib/commons-codec-1.4.jar:/opt/hbase/bin/../lib/commons-collections-3.2.1.jar:/opt/hbase/bin/../lib/commons-configuration-1.6.jar:/opt/hbase/bin/../lib/commons-digester-1.8.jar:/opt/hbase/bin/../lib/commons-el-1.0.jar:/opt/hbase/bin/../lib/commons-httpclient-3.1.jar:/opt/hbase/bin/../lib/commons-io-2.1.jar:/opt/hbase/bin/../lib/commons-lang-2.5.jar:/opt/hbase/bin/../lib/commons-logging-1.1.1.jar:/opt/hbase/bin/../lib/commons-math-2.1.jar:/opt/hbase/bin/../lib/commons-net-1.4.1.jar:/opt/hbase/bin/../lib/core-3.1.1.jar:/opt/hbase/bin/../lib/guava-11.0.2.jar:/opt/hbase/bin/../lib/hadoop-core-1.0.3.jar:/opt/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/opt/hbase/bin/../lib/httpclient-4.1.2.jar:/opt/hbase/bin/../lib/httpcore-4.1.3.jar:/opt/hbase/bin/../lib/jackson-core-asl-1.8.8.jar:/opt/hbase/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/hbase/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/hbase/bin/../lib/jackson-xc-1.8.8.jar:/opt/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/opt/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/opt/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/opt/hbase/bin/../lib/jaxb-api-2.1.jar:/opt/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/bin/../lib/jersey-core-1.8.jar:/opt/hbase/bin/../lib/jersey-json-1.8.jar:/opt/hbase/bin/../lib/jersey-server-1.8.jar:/opt/hbase/bin/../lib/jettison-1.1.jar:/opt/hbase/bin/../lib/jetty-6.1.26.jar:/opt/hbase/bin/../lib/jetty-util-6.1.26.jar:/opt/hbase/bin/../lib/jruby-complete-1.6.5.jar:/opt/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/bin/../lib/jsr305-1.3.9.jar:/opt/hbase/bin/../lib/junit-4.10-HBASE-1.jar:/opt/hbase/bin/../lib/libthrift-0.8.0.jar:/opt/hbase/bin/../lib/log4j-1.2.16.jar:/opt/hbase/bin/../lib/metrics-core-2.1.2.jar:/opt/hbase/bin/../lib/netty-3.2.4.Final.jar:/opt/hbase/bin/../lib/protobuf-java-2.4.0a.jar:/opt/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/bin/../lib/slf4j-api-1.4.3.jar:/opt/hbase/bin/../lib/slf4j-log4j12-1.4.3.jar:/opt/hbase/bin/../lib/snappy-java-1.0.3.2.jar:/opt/hbase/bin/../lib/stax-api-1.0.1.jar:/opt/hbase/bin/../lib/velocity-1.7.jar:/opt/hbase/bin/../lib/xmlenc-0.52.jar:/opt/hbase/bin/../lib/zookeeper-3.4.3.jar:/opt/hbase/bin/../libextra/mybk-commons-cc.jar:/opt/hbase/bin/../libextra/hbase.jar:/opt/hbase/bin/../libextra/sfdcloud-hbase.jar: 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/hbase/bin/../lib/native/Linux-i386-32 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:os.arch=i386 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-33-generic 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:user.name=pcer 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/pcer 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Client environment:user.dir=/opt/hbase-0.94.1 12/11/08 19:20:40 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=dev-vm0:2181 sessionTimeout=60000 watcher=hconnection 12/11/08 19:20:40 INFO zookeeper.ClientCnxn: Opening socket connection to server /172.16.20.140:2181 12/11/08 19:20:40 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 28728@dev-vm0 12/11/08 19:20:40 WARN client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration. 12/11/08 19:20:40 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 12/11/08 19:20:40 INFO zookeeper.ClientCnxn: Socket connection established to dev-vm0/172.16.20.140:2181, initiating session 12/11/08 19:20:40 INFO zookeeper.ClientCnxn: Session establishment complete on server dev-vm0/172.16.20.140:2181, sessionid = 0x13ad3e9ba700150, negotiated timeout = 40000 12/11/08 19:20:40 DEBUG client.HConnectionManager$HConnectionImplementation: Looked up root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a; serverName=dev-vm0,46871,1352175042380 12/11/08 19:20:41 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for .META.,,1.1028785192 is dev-vm0:46871 12/11/08 19:20:41 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,,00000000000000 for max=10 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:20:41 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for test2,,1352373607304.8341524d6c8b105b1722961ebda3a048. is dev-vm0:46871 12/11/08 19:20:41 DEBUG util.RegionSplitter: No _balancedSplit file. Calculating splits... 12/11/08 19:20:41 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,,00000000000000 for max=2147483647 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:20:41 DEBUG util.RegionSplitter: Table test2 has 1 regions that will be split. 12/11/08 19:20:41 DEBUG util.RegionSplitter: Will Split [00000000 , ffffffff) at 7fffffff 12/11/08 19:20:41 DEBUG util.RegionSplitter: Bucketing regions by regionserver... 12/11/08 19:20:41 DEBUG util.RegionSplitter: Done with bucketing. Split time! 12/11/08 19:20:41 DEBUG util.RegionSplitter: 1 RS have regions to splt. 12/11/08 19:20:41 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,,00000000000000 for max=2147483647 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:20:41 DEBUG util.RegionSplitter: Finding a region on dev-vm0:46871 12/11/08 19:20:41 DEBUG util.RegionSplitter: Splitting at 7fffffff 12/11/08 19:20:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=dev-vm0:2181 sessionTimeout=60000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:20:41 INFO zookeeper.ClientCnxn: Opening socket connection to server /172.16.20.140:2181 12/11/08 19:20:41 WARN client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration. 12/11/08 19:20:41 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 12/11/08 19:20:41 INFO zookeeper.ClientCnxn: Socket connection established to dev-vm0/172.16.20.140:2181, initiating session 12/11/08 19:20:41 INFO zookeeper.ClientCnxn: Session establishment complete on server dev-vm0/172.16.20.140:2181, sessionid = 0x13ad3e9ba700151, negotiated timeout = 40000 12/11/08 19:20:41 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 28728@dev-vm0 12/11/08 19:20:41 DEBUG catalog.CatalogTracker: Starting catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@29c58e 12/11/08 19:20:41 DEBUG client.ClientScanner: Creating scanner over .META. starting at key 'test2,,' 12/11/08 19:20:41 DEBUG client.ClientScanner: Advancing internal scanner to startKey at 'test2,,' 12/11/08 19:20:41 DEBUG client.ClientScanner: Creating scanner over .META. starting at key 'test2,,' 12/11/08 19:20:41 DEBUG client.ClientScanner: Advancing internal scanner to startKey at 'test2,,' 12/11/08 19:20:41 DEBUG client.ClientScanner: Finished with scanning at {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192,} 12/11/08 19:20:41 DEBUG catalog.CatalogTracker: Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@29c58e 12/11/08 19:20:41 INFO zookeeper.ZooKeeper: Session: 0x13ad3e9ba700151 closed 12/11/08 19:20:41 INFO zookeeper.ClientCnxn: EventThread shut down 12/11/08 19:20:41 DEBUG client.HConnectionManager$HConnectionImplementation: Looked up root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a; serverName=dev-vm0,46871,1352175042380 12/11/08 19:20:41 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for .META.,,1.1028785192 is dev-vm0:46871 12/11/08 19:20:41 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,,00000000000000 for max=10 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:20:41 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for test2,,1352373607304.8341524d6c8b105b1722961ebda3a048. is dev-vm0:46871 12/11/08 19:20:41 DEBUG util.RegionSplitter: Split Scan: 0 finished / 1 split wait / 0 reference wait 12/11/08 19:21:11 DEBUG client.HConnectionManager$HConnectionImplementation: Looked up root region location, connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a; serverName=dev-vm0,46871,1352175042380 12/11/08 19:21:11 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for .META.,,1.1028785192 is dev-vm0:46871 12/11/08 19:21:11 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,7fffffff,00000000000000 for max=10 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:21:11 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for test2,7fffffff,1352373641571.d56fbb146b77bbc87c294766fabcb3c4. is dev-vm0:46871 12/11/08 19:21:11 DEBUG client.MetaScanner: Scanning .META. starting at row=test2,,00000000000000 for max=10 rows using org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a 12/11/08 19:21:11 DEBUG client.HConnectionManager$HConnectionImplementation: Cached location for test2,,1352373641571.4e805892dd68d69ba02f72303973ee0e. is dev-vm0:46871 12/11/08 19:21:12 DEBUG util.RegionSplitter: Split Scan: 1 finished / 0 split wait / 0 reference wait 12/11/08 19:21:12 DEBUG util.RegionSplitter: All regions have been successfully split! 12/11/08 19:21:12 DEBUG util.RegionSplitter: TOTAL TIME = 30sec 12/11/08 19:21:12 DEBUG util.RegionSplitter: Splits = 0 Exception in thread "main" java.lang.ArithmeticException: / by zero at org.apache.hadoop.hbase.util.RegionSplitter.rollingSplit(RegionSplitter.java:576) at org.apache.hadoop.hbase.util.RegionSplitter.main(RegionSplitter.java:349)
The attached patch was tested and worked well.