[FLUME-2134] AsyncHbase Sink bugfix plus tests errors on Windows - ASF JIRA

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 1.4.0
Fix Version/s: 1.4.1, 1.5.0
Component/s: Sinks+Sources, Test, Windows
Labels:
- hbase
- sink
- test
- windows

Release Note:
Fixes for AsyncHbase sink to run on Windows

Description

Problem appears to be inabilty to handle windows paths.

13/07/23 17:16:35 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x1400e08d22b0000 type:delete cxid:0x13 zxid:0xb txntype:-1 reqpath:n/a Error Path:/hbase/backup-masters/192.168.50.128,55884,1374624994459 Error:KeeperErrorCode = NoNode for /hbase/backup-masters/192.168.50.128,55884,1374624994459
13/07/23 17:16:35 WARN zookeeper.RecoverableZooKeeper: Node /hbase/backup-masters/192.168.50.128,55884,1374624994459 already deleted, and this is not a retry
13/07/23 17:16:35 INFO master.ActiveMasterManager: Master=192.168.50.128,55884,1374624994459
13/07/23 17:16:35 ERROR master.HMaster: Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
	at org.apache.hadoop.fs.Path.initialize(Path.java:148)
	at org.apache.hadoop.fs.Path.<init>(Path.java:71)
	at org.apache.hadoop.fs.Path.<init>(Path.java:50)
	at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:111)
	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:532)
	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:391)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
	at java.net.URI.checkPath(URI.java:1804)
	at java.net.URI.<init>(URI.java:752)
	at org.apache.hadoop.fs.Path.initialize(Path.java:145)
	... 6 more
13/07/23 17:16:35 INFO master.HMaster: Aborting
13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping server on 55884
13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server listener on 55884
13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 0 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 1 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 2 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 3 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 4 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 5 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 6 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 7 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 8 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 9 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 0 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 1 on 55884: exiting
13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 2 on 55884: exiting
13/07/23 17:16:35 INFO server.PrepRequestProcessor: Processed session termination for sessionid: 0x1400e08d22b0000
13/07/23 17:16:35 INFO server.NIOServerCnxn: Closed socket connection for client /192.168.50.128:55911 which had sessionid 0x1400e08d22b0000
13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Session: 0x1400e08d22b0000 closed
13/07/23 17:16:35 INFO master.HMaster: HMaster main thread exiting
13/07/23 17:16:35 INFO zookeeper.ClientCnxn: EventThread shut down
13/07/23 17:16:35 INFO regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999
13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=0.0.0.0:2181 sessionTimeout=180000 watcher=regionserver:55912
13/07/23 17:16:35 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 5824@WIN-6AKNOG7MFK5
13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Opening socket connection to server 0.0.0.0/0.0.0.0:2181. Will not attempt to authenticate using SASL (unknown error)
13/07/23 17:16:35 INFO server.NIOServerCnxnFactory: Accepted socket connection from /192.168.50.128:55939
13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Socket connection established to 0.0.0.0/0.0.0.0:2181, initiating session
13/07/23 17:16:35 INFO server.ZooKeeperServer: Client attempting to establish new session at /192.168.50.128:55939
13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Session establishment complete on server 0.0.0.0/0.0.0.0:2181, sessionid = 0x1400e08d22b0001, negotiated timeout = 40000
13/07/23 17:16:35 INFO server.ZooKeeperServer: Established session 0x1400e08d22b0001 with negotiated timeout 40000 for client /192.168.50.128:55939
13/07/23 17:19:55 ERROR hbase.MiniHBaseCluster: Error starting cluster
java.lang.RuntimeException: Master not initialized after 200 seconds
	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:206)
	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:420)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:216)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
13/07/23 17:19:55 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
13/07/23 17:19:55 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
13/07/23 17:19:57 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
13/07/23 17:19:57 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
13/07/23 17:20:01 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
13/07/23 17:20:01 INFO util.RetryCounter: Sleeping 8000ms before retry #3...
13/07/23 17:20:09 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
13/07/23 17:20:09 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 3 retries
13/07/23 17:20:09 ERROR master.HMaster: ZooKeeper exception trying to set cluster as down in ZK
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:133)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1197)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1186)
	at org.apache.hadoop.hbase.zookeeper.ClusterStatusTracker.setClusterDown(ClusterStatusTracker.java:80)
	at org.apache.hadoop.hbase.master.HMaster.shutdown(HMaster.java:1672)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:234)
	at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
	at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
13/07/23 17:20:09 INFO regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-19,5,main]
13/07/23 17:20:09 ERROR server.NIOServerCnxnFactory: Thread Thread[Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999,5,main] died
java.lang.NullPointerException
	at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
	at org.apache.hadoop.hbase.regionserver.ShutdownHook$ShutdownHookThread.run(ShutdownHook.java:113)

java.lang.NullPointerException
	at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:241)
	at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
	at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)


java.lang.NullPointerException
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)


java.lang.NullPointerException
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)


java.lang.NullPointerException
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)


java.lang.NullPointerException
	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)

Disconnected from the target VM, address: '127.0.0.1:55877', transport: 'socket'

Process finished with exit code -1

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

FLUME-2134.patch
24/Jul/13 03:03
6 kB
Roshan Naik
FLUME-2134.v2.patch
05/Aug/13 18:53
6 kB
Roshan Naik
FLUME-2134.v3.patch
05/Aug/13 23:53
7 kB
Roshan Naik

Issue Links

links to

Reviewboard

AsyncHbase Sink bugfix plus tests errors on Windows

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates