Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-7934

OOM due to LibeventSSLSocket send incorrectly returning 0 after shutdown.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.3, 1.3.2, 1.4.0
    • Component/s: libprocess
    • Labels:
      None

      Description

      LibeventSSLSocket can return 0 from send incorrectly, which leads the caller to send the data twice!

      See here: https://github.com/apache/mesos/blob/1.3.1/3rdparty/libprocess/src/libevent_ssl_socket.cpp#L396-L398

      In some particular cases, it's possible that the caller keeps getting back 0 and loops infinitely, blowing up the memory and OOMing the process.

      One example is when a send occurs after a shutdown:

      TEST_F(SSLTest, ShutdownThenSend)
      {
        Clock::pause();
      
        Try<Socket> server = setup_server({
            {"LIBPROCESS_SSL_ENABLED", "true"},
            {"LIBPROCESS_SSL_KEY_FILE", key_path().string()},
            {"LIBPROCESS_SSL_CERT_FILE", certificate_path().string()}});
      
        ASSERT_SOME(server);
        ASSERT_SOME(server.get().address());
        ASSERT_SOME(server.get().address().get().hostname());
      
        Future<Socket> socket = server.get().accept();
      
        Clock::settle();
        EXPECT_TRUE(socket.isPending());
      
        Try<Socket> client = Socket::create(SocketImpl::Kind::SSL);
        ASSERT_SOME(client);
        AWAIT_ASSERT_READY(client->connect(server->address().get()));
      
        AWAIT_ASSERT_READY(socket);
      
        EXPECT_SOME(Socket(socket.get()).shutdown());
      
        // This loops forever!
        AWAIT_FAILED(Socket(socket.get()).send("Hello World"));
      }
      

        Attachments

          Activity

            People

            • Assignee:
              bmahler Benjamin Mahler
              Reporter:
              bmahler Benjamin Mahler
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: