Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2452

Unit test TestAvroSink.java failed with IBM JDK 1.7

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 1.5.0.1
    • Fix Version/s: None
    • Component/s: Sinks+Sources, Test
    • Labels:
    • Environment:

      RHEL 6.5 PPC64 & x86_64
      IBM JDK 1.7.0

      Description

      Running org.apache.flume.sink.TestAvroSink

      Tests run: 15, Failures: 0, Errors: 9, Skipped: 0, Time elapsed: 41.619 sec <<< FAILURE!
      testSslProcessWithTrustStore(org.apache.flume.sink.TestAvroSink) Time elapsed: 31 sec <<< ERROR!

      Bar.java
      org.apache.flume.EventDeliveryException: Failed to send events
              \\at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
              at org.apache.flume.sink.TestAvroSink.testSslProcessWithTrustStore(TestAvroSink.java:431)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
              at java.lang.reflect.Method.invoke(Method.java:619)
              at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
              at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
              at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
              at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
              at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
              at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
              at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
              at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
              at java.lang.reflect.Method.invoke(Method.java:619)
              at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
              at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
              at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
       at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
              at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
      Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host: localhost, port: 41414 }: Unexpected exception
              at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:184)
              at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:118)
              at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:624)
              at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:88)
              at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)
              at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:211)
              at org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:272)
              at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:349)
              ... 30 more
      Caused by: java.lang.RuntimeException: Cannot create SSL channel
              at org.apache.flume.api.NettyAvroRpcClient$SSLCompressionChannelFactory.newChannel(NettyAvroRpcClient.java:747)
              at org.apache.flume.api.NettyAvroRpcClient$SSLCompressionChannelFactory.newChannel(NettyAvroRpcClient.java:663)
              at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:212)
              at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182)
              at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:253)
              at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)
              at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)
              at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:164)
              ... 37 more
      Caused by: java.security.NoSuchAlgorithmException: SunX509 TrustManagerFactory not available
              at sun.security.jca.GetInstance.getInstance(GetInstance.java:171)
              at javax.net.ssl.TrustManagerFactory.getInstance(TrustManagerFactory.java:6)
              at org.apache.flume.api.NettyAvroRpcClient$SSLCompressionChannelFactory.newChannel(NettyAvroRpcClient.java:727)
              ... 44 more
      

      In file ./flume-ng-sdk/src/main/java/org/apache/flum/api/NettyAvroRpcClient.java, the TrustManagerFactory is de facto SunX509

      TrustManagerFactory tmf = TrustManagerFactory.getInstance("SunX509");

      I think there should be a change, like the one present in the ./flume-ng-core/src/test/java/org/apache/flume/sink/TestAvroSink.java file :

      Bar.java
      private String getAlgorithm()
      {
            String algorithm = Security.getProperty(
                "ssl.KeyManagerFactory.algorithm");
            if (algorithm == null) {
              algorithm = "SunX509";
            }
            return algorithm;
      }
      

      Even though it gets rid of the error "SunX509 TrustManagerFactory not available" when applied, I then get another error in place of it :

      Bar.java
      org.apache.flume.EventDeliveryException: Failed to send events
              at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:392)
              at org.apache.flume.sink.TestAvroSink.testSslProcessWithTrustStore(TestAvroSink.java:431)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
              at java.lang.reflect.Method.invoke(Method.java:619)
              at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
              at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
              at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
              at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
              at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
              at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
              at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
              at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
              at java.lang.reflect.Method.invoke(Method.java:619)
              at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
              at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
              at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
              at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
              at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
      Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 41414 }: Failed to send batch
              at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:312)
              at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:376)
              ... 30 more
      Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: localhost, port: 41414 }: Exception thrown from remote handler
              at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:394)
              at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:371)
              at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:300)
              ... 31 more
      Caused by: java.util.concurrent.ExecutionException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
              at org.apache.avro.ipc.CallFuture.get(CallFuture.java:128)
              at org.apache.flume.api.NettyAvroRpcClient.waitForStatusOK(NettyAvroRpcClient.java:386)
              ... 33 more
      Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
              at com.ibm.jsse2.ab.y(ab.java:364)
              at com.ibm.jsse2.nc.b(nc.java:461)
              at com.ibm.jsse2.nc.a(nc.java:283)
              at com.ibm.jsse2.nc.unwrap(nc.java:333)
              at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:26)
              at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1118)
              at org.jboss.netty.handler.ssl.SslHandler.decode(SslHandler.java:814)
              at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:422)
              at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
              at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
              at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
              at java.lang.Thread.run(Thread.java:853)
      Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
              at com.ibm.jsse2.j.a(j.java:4)
              at com.ibm.jsse2.nc.a(nc.java:524)
              at com.ibm.jsse2.ab.a(ab.java:166)
              at com.ibm.jsse2.ab.a(ab.java:534)
              at com.ibm.jsse2.bb.a(bb.java:587)
              at com.ibm.jsse2.bb.a(bb.java:472)
              at com.ibm.jsse2.ab.r(ab.java:489)
              at com.ibm.jsse2.ab$1.a(ab$1.java:3)
              at com.ibm.jsse2.ab$1.run(ab$1.java:1)
              at java.security.AccessController.doPrivileged(AccessController.java:366)
              at com.ibm.jsse2.ab$c_.run(ab$c_.java:11)
              at org.jboss.netty.handler.ssl.SslHandler$2.run(SslHandler.java:1258)
              at org.jboss.netty.handler.ssl.ImmediateExecutor.execute(ImmediateExecutor.java:31)
              at org.jboss.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1255)
              at org.jboss.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1140)
              ... 12 more
      Caused by: com.ibm.jsse2.util.j: Certificate expired
              at com.ibm.jsse2.util.i.b(i.java:175)
              at com.ibm.jsse2.util.g.a(g.java:10)
              at com.ibm.jsse2.yc.a(yc.java:141)
              at com.ibm.jsse2.yc.a(yc.java:124)
              at com.ibm.jsse2.yc.checkServerTrusted(yc.java:43)
              at com.ibm.jsse2.bb.a(bb.java:231)
              ... 22 more
      

      I do not understand this "certificate expired" exception.
      In addition, others tests from TestAvroSink are failing because they try to bind to the same port this test was bind on, leading to "Failed to bind to: 127.0.0.1:41414" message

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Mickael Olivier Mickael Olivier

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment