Flume
  1. Flume
  2. FLUME-2134

AsyncHbase Sink bugfix plus tests errors on Windows

    Details

    • Release Note:
      Fixes for AsyncHbase sink to run on Windows

      Description

      Problem appears to be inabilty to handle windows paths.

      13/07/23 17:16:35 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x1400e08d22b0000 type:delete cxid:0x13 zxid:0xb txntype:-1 reqpath:n/a Error Path:/hbase/backup-masters/192.168.50.128,55884,1374624994459 Error:KeeperErrorCode = NoNode for /hbase/backup-masters/192.168.50.128,55884,1374624994459
      13/07/23 17:16:35 WARN zookeeper.RecoverableZooKeeper: Node /hbase/backup-masters/192.168.50.128,55884,1374624994459 already deleted, and this is not a retry
      13/07/23 17:16:35 INFO master.ActiveMasterManager: Master=192.168.50.128,55884,1374624994459
      13/07/23 17:16:35 ERROR master.HMaster: Unhandled exception. Starting shutdown.
      java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
      	at org.apache.hadoop.fs.Path.initialize(Path.java:148)
      	at org.apache.hadoop.fs.Path.<init>(Path.java:71)
      	at org.apache.hadoop.fs.Path.<init>(Path.java:50)
      	at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:111)
      	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:532)
      	at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:391)
      	at java.lang.Thread.run(Thread.java:722)
      Caused by: java.net.URISyntaxException: Relative path in absolute URI: file://C:%5CUsers%5CROSHAN~1%5CAppData%5CLocal%5CTemp%5C1374624993647-0%5Chbase.tmp
      	at java.net.URI.checkPath(URI.java:1804)
      	at java.net.URI.<init>(URI.java:752)
      	at org.apache.hadoop.fs.Path.initialize(Path.java:145)
      	... 6 more
      13/07/23 17:16:35 INFO master.HMaster: Aborting
      13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping server on 55884
      13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server listener on 55884
      13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
      13/07/23 17:16:35 INFO ipc.HBaseServer: Stopping IPC Server Responder
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 0 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 1 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 2 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 3 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 4 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 5 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 6 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 7 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 8 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: IPC Server handler 9 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 0 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 1 on 55884: exiting
      13/07/23 17:16:35 INFO ipc.HBaseServer: REPL IPC Server handler 2 on 55884: exiting
      13/07/23 17:16:35 INFO server.PrepRequestProcessor: Processed session termination for sessionid: 0x1400e08d22b0000
      13/07/23 17:16:35 INFO server.NIOServerCnxn: Closed socket connection for client /192.168.50.128:55911 which had sessionid 0x1400e08d22b0000
      13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Session: 0x1400e08d22b0000 closed
      13/07/23 17:16:35 INFO master.HMaster: HMaster main thread exiting
      13/07/23 17:16:35 INFO zookeeper.ClientCnxn: EventThread shut down
      13/07/23 17:16:35 INFO regionserver.ShutdownHook: Installed shutdown hook thread: Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999
      13/07/23 17:16:35 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=0.0.0.0:2181 sessionTimeout=180000 watcher=regionserver:55912
      13/07/23 17:16:35 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 5824@WIN-6AKNOG7MFK5
      13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Opening socket connection to server 0.0.0.0/0.0.0.0:2181. Will not attempt to authenticate using SASL (unknown error)
      13/07/23 17:16:35 INFO server.NIOServerCnxnFactory: Accepted socket connection from /192.168.50.128:55939
      13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Socket connection established to 0.0.0.0/0.0.0.0:2181, initiating session
      13/07/23 17:16:35 INFO server.ZooKeeperServer: Client attempting to establish new session at /192.168.50.128:55939
      13/07/23 17:16:35 INFO zookeeper.ClientCnxn: Session establishment complete on server 0.0.0.0/0.0.0.0:2181, sessionid = 0x1400e08d22b0001, negotiated timeout = 40000
      13/07/23 17:16:35 INFO server.ZooKeeperServer: Established session 0x1400e08d22b0001 with negotiated timeout 40000 for client /192.168.50.128:55939
      13/07/23 17:19:55 ERROR hbase.MiniHBaseCluster: Error starting cluster
      java.lang.RuntimeException: Master not initialized after 200 seconds
      	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:206)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:420)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:216)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:601)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      13/07/23 17:19:55 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
      13/07/23 17:19:55 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
      13/07/23 17:19:57 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
      13/07/23 17:19:57 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
      13/07/23 17:20:01 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
      13/07/23 17:20:01 INFO util.RetryCounter: Sleeping 8000ms before retry #3...
      13/07/23 17:20:09 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
      13/07/23 17:20:09 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 3 retries
      13/07/23 17:20:09 ERROR master.HMaster: ZooKeeper exception trying to set cluster as down in ZK
      org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/shutdown
      	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
      	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
      	at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
      	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:133)
      	at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1197)
      	at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1186)
      	at org.apache.hadoop.hbase.zookeeper.ClusterStatusTracker.setClusterDown(ClusterStatusTracker.java:80)
      	at org.apache.hadoop.hbase.master.HMaster.shutdown(HMaster.java:1672)
      	at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:234)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:601)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      13/07/23 17:20:09 INFO regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=Thread[Thread-19,5,main]
      13/07/23 17:20:09 ERROR server.NIOServerCnxnFactory: Thread Thread[Shutdownhook:RegionServer:0;192.168.50.128,55912,1374624994999,5,main] died
      java.lang.NullPointerException
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
      	at org.apache.hadoop.hbase.regionserver.ShutdownHook$ShutdownHookThread.run(ShutdownHook.java:113)
      
      java.lang.NullPointerException
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.stop(HRegionServer.java:1810)
      	at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:241)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:427)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.shutdown(MiniHBaseCluster.java:492)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:86)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:77)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:64)
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:123)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      
      
      java.lang.NullPointerException
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      
      
      java.lang.NullPointerException
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      
      
      java.lang.NullPointerException
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      
      
      java.lang.NullPointerException
      	at org.apache.flume.sink.hbase.TestAsyncHBaseSink.tearDown(TestAsyncHBaseSink.java:162)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
      
      Disconnected from the target VM, address: '127.0.0.1:55877', transport: 'socket'
      
      Process finished with exit code -1
      
      1. FLUME-2134.v3.patch
        7 kB
        Roshan Naik
      2. FLUME-2134.v2.patch
        6 kB
        Roshan Naik
      3. FLUME-2134.patch
        6 kB
        Roshan Naik

        Issue Links

          Activity

          Hide
          Roshan Naik added a comment -

          Planning to use testUtility to allocate HBase+ZK cluster instead of explicitly allocating ZK and HBase clusters... similar to how to its done in TestHBaseSink.

          Show
          Roshan Naik added a comment - Planning to use testUtility to allocate HBase+ZK cluster instead of explicitly allocating ZK and HBase clusters... similar to how to its done in TestHBaseSink.
          Hide
          Roshan Naik added a comment -

          The Async HBase sink uses conf.get("hbase.zookeeper.quorum") to get the zookeeper quorum info... which on windows is only returning "localhost". HBase folks advised the use of ZKConfig.getZKQuorumServersString(conf) instead. I confirmed it returns the right value "localhost:port#" on both Windows and Linux.

          Show
          Roshan Naik added a comment - The Async HBase sink uses conf.get("hbase.zookeeper.quorum") to get the zookeeper quorum info... which on windows is only returning "localhost". HBase folks advised the use of ZKConfig.getZKQuorumServersString(conf) instead. I confirmed it returns the right value "localhost:port#" on both Windows and Linux.
          Hide
          Roshan Naik added a comment -

          Patch contains:

          • Bug fix for getting ZK quorum in Async HBase sink
          • Simplified the setup code in the sink's test to fix a problem execution error on Windows.
          Show
          Roshan Naik added a comment - Patch contains: Bug fix for getting ZK quorum in Async HBase sink Simplified the setup code in the sink's test to fix a problem execution error on Windows.
          Hide
          Hari Shreedharan added a comment -

          Hi Roshan,

          This patch seems to have increased the time that AsyncHBase test runs to about 15 mins, while it took like 1 min earlier:

          Without the patch:

          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestAsyncHBaseSink
          2013-07-29 15:07:27.132 java[26397:1203] Unable to load realm info from SCDynamicStore
          Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 47.948 sec
          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestHBaseSink
          2013-07-29 15:08:15.574 java[26401:1203] Unable to load realm info from SCDynamicStore
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 48.788 sec
          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestHBaseSinkCreation
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec
          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestRegexHbaseEventSerializer
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.357 sec
          
          

          With the patch:

          
          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestAsyncHBaseSink
          2013-07-29 14:38:23.135 java[24708:1203] Unable to load realm info from SCDynamicStore
          Picked up _JAVA_OPTIONS: -Djava.awt.headless=true
          Running org.apache.flume.sink.hbase.TestHBaseSink
          2013-07-29 14:53:24.328 java[25900:1203] Unable to load realm info from SCDynamicStore
          
          

          Could you take a look?

          Show
          Hari Shreedharan added a comment - Hi Roshan, This patch seems to have increased the time that AsyncHBase test runs to about 15 mins, while it took like 1 min earlier: Without the patch: Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestAsyncHBaseSink 2013-07-29 15:07:27.132 java[26397:1203] Unable to load realm info from SCDynamicStore Tests run: 8, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 47.948 sec Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestHBaseSink 2013-07-29 15:08:15.574 java[26401:1203] Unable to load realm info from SCDynamicStore Tests run: 6, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 48.788 sec Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestHBaseSinkCreation Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.234 sec Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestRegexHbaseEventSerializer Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.357 sec With the patch: ------------------------------------------------------- T E S T S ------------------------------------------------------- Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestAsyncHBaseSink 2013-07-29 14:38:23.135 java[24708:1203] Unable to load realm info from SCDynamicStore Picked up _JAVA_OPTIONS: -Djava.awt.headless= true Running org.apache.flume.sink.hbase.TestHBaseSink 2013-07-29 14:53:24.328 java[25900:1203] Unable to load realm info from SCDynamicStore Could you take a look?
          Hide
          Roshan Naik added a comment -

          Thanks. The problem is in the testWithoutConfigurationObject(). It also needs the same fix to use ZKConfig.getZKQuorumServersString()

          Show
          Roshan Naik added a comment - Thanks. The problem is in the testWithoutConfigurationObject(). It also needs the same fix to use ZKConfig.getZKQuorumServersString()
          Hide
          Roshan Naik added a comment -

          one line change to patch to use ZKConfig.getZKQuorumServersString() in testWithoutConfigurationObject

          Show
          Roshan Naik added a comment - one line change to patch to use ZKConfig.getZKQuorumServersString() in testWithoutConfigurationObject
          Hide
          Hari Shreedharan added a comment -

          Roshan,

          Looks good. There are several unused imports in the test. Could you please clean those up? I will commit it once that is done. Thanks.

          Show
          Hari Shreedharan added a comment - Roshan, Looks good. There are several unused imports in the test. Could you please clean those up? I will commit it once that is done. Thanks.
          Hide
          Roshan Naik added a comment - - edited

          removing imports that are no longer used

          Show
          Roshan Naik added a comment - - edited removing imports that are no longer used
          Hide
          Hari Shreedharan added a comment -

          +1.

          Show
          Hari Shreedharan added a comment - +1.
          Hide
          Hari Shreedharan added a comment -

          Committed, rev: 5d49eeb734d1c55aaf48219699706851d9e820b5. Thanks roshan!

          Show
          Hari Shreedharan added a comment - Committed, rev: 5d49eeb734d1c55aaf48219699706851d9e820b5. Thanks roshan!
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in flume-trunk #485 (See https://builds.apache.org/job/flume-trunk/485/)
          FLUME-2134. AsyncHbase Sink should use ZKConfig.getZKQuorumServersString plus test fixes on Windows (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=5d49eeb734d1c55aaf48219699706851d9e820b5)

          • flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestAsyncHBaseSink.java
          • flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java
          Show
          Hudson added a comment - SUCCESS: Integrated in flume-trunk #485 (See https://builds.apache.org/job/flume-trunk/485/ ) FLUME-2134 . AsyncHbase Sink should use ZKConfig.getZKQuorumServersString plus test fixes on Windows (hshreedharan: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=5d49eeb734d1c55aaf48219699706851d9e820b5 ) flume-ng-sinks/flume-ng-hbase-sink/src/test/java/org/apache/flume/sink/hbase/TestAsyncHBaseSink.java flume-ng-sinks/flume-ng-hbase-sink/src/main/java/org/apache/flume/sink/hbase/AsyncHBaseSink.java

            People

            • Assignee:
              Roshan Naik
              Reporter:
              Roshan Naik
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development