Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2519

Use NettyTransceiver repeatedly connect to remote server which doesn't support current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct buffer memory"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.9.0
    • 1.10.1
    • java
    • None
    • java version "12.0.1" 2019-04-16

      Java(TM) SE Runtime Environment (build 12.0.1+12)

      Java HotSpot(TM) 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)

    Description

      I just modify the code from https://github.com/phunt/avro-rpc-quickstart。Execute the modified code can 100% reproduce this issue. I have attached the test project file. When use NettyTransceiver repeatedly connect to remote server which doesn't support current RPC or unavailable will cause " java.lang.OutOfMemoryError: Direct buffer memory"

      The source to execute:

      /**

       * Licensed to the Apache Software Foundation (ASF) under one

       * or more contributor license agreements.  See the NOTICE file

       * distributed with this work for additional information

       * regarding copyright ownership.  The ASF licenses this file

       * to you under the Apache License, Version 2.0 (the

       * "License"); you may not use this file except in compliance

       * with the License.  You may obtain a copy of the License at

       *

       *     http://www.apache.org/licenses/LICENSE-2.0

       *

       * Unless required by applicable law or agreed to in writing, software

       * distributed under the License is distributed on an "AS IS" BASIS,

       * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

       * See the License for the specific language governing permissions and

       * limitations under the License.

       */

       

      package example;

       

      import java.io.IOException;

      import java.net.InetAddress;

      import java.net.InetSocketAddress;

       

      import org.apache.avro.ipc.netty.NettyServer;

      import org.apache.avro.ipc.netty.NettyTransceiver;

      import org.apache.avro.ipc.Server;

      import org.apache.avro.ipc.specific.SpecificRequestor;

      import org.apache.avro.ipc.specific.SpecificResponder;

      import org.apache.avro.util.Utf8;

       

      import example.proto.Mail;

      import example.proto.Message;

       

      /**

       * Start a server, attach a client, and send a message.

       */

      public class Main {

      public static class MailImpl implements Mail {

      // in this simple example just return details of the message

      public Utf8 send(Message message)

      { System.out.println("Sending message"); return new Utf8("Sending message to " + message.getTo().toString() + " from " + message.getFrom().toString() + " with body " + message.getBody().toString()); }

      }

       

      private static Server server;

       

      private static void startServer() throws IOException

      { server = new NettyServer(new SpecificResponder(Mail.class, new MailImpl()), new InetSocketAddress(65111)); // the server implements the Mail protocol (MailImpl) }

       

      public static void main(String[] args) throws IOException {

      //        if (args.length != 3)

      { //            System.out.println("Usage: <to> <from> <body>"); //            System.exit(1); //        }

       

      System.out.println("Starting server");

      // usually this would be another app, but for simplicity

      //        startServer();

      System.out.println("Server started");

       

      while (true) {

      NettyTransceiver client = null;

      try

      { client = new NettyTransceiver(new InetSocketAddress(InetAddress.getByName("129.28.206.27"), 7999), 3000l); // client code - attach to the server and send a message Mail proxy = (Mail) SpecificRequestor.getClient(Mail.class, client); System.out.println("Client built, got proxy");   // fill in the Message record and send it Message message = new Message(); message.setTo(new Utf8(args[0])); message.setFrom(new Utf8(args[1])); message.setBody(new Utf8(args[2])); System.out.println("Calling proxy.send with message:  " + message.toString()); System.out.println("Result: " + proxy.send(message)); }

      catch (Exception e)

      { System.out.println(e.getMessage()); }

      finally {

      // cleanup

      if(client != null)

      { client.close(); }

      }

       

      }

       

      //        server.close();

      }

      }

      The error log:

      Starting server

      Server started

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Error connecting to /129.28.206.27:7999

      Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory

      at java.base/java.nio.Bits.reserveMemory(Bits.java:175)

      at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)

      at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)

      at org.jboss.netty.channel.socket.nio.SocketSendBufferPool$Preallocation.<init>(SocketSendBufferPool.java:156)

      at org.jboss.netty.channel.socket.nio.SocketSendBufferPool.<init>(SocketSendBufferPool.java:42)

      at org.jboss.netty.channel.socket.nio.AbstractNioWorker.<init>(AbstractNioWorker.java:45)

      at org.jboss.netty.channel.socket.nio.NioWorker.<init>(NioWorker.java:45)

      at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:44)

      at org.jboss.netty.channel.socket.nio.NioWorkerPool.newWorker(NioWorkerPool.java:28)

      at org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:80)

      at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:39)

      at org.jboss.netty.channel.socket.nio.NioWorkerPool.<init>(NioWorkerPool.java:33)

      at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:151)

      at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.<init>(NioClientSocketChannelFactory.java:116)

      at org.apache.avro.ipc.netty.NettyTransceiver.<init>(NettyTransceiver.java:127)

      at example.Main.main(Main.java:69)

       

      The JVM parameter:

      -XX:MaxDirectMemorySize=8m -Dio.netty.leakDetection.level=advanced

       

      Attachments

        Issue Links

          Activity

            People

              julian.wexler Julian Wexler
              nju2006 Xun Wang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: