[THRIFT-4404] Is TSSLSocket is thred-safe in thift version 0.9.3?

• Type: Question
• Status: Closed
• Priority: Major
• Resolution: Information Provided
• Affects Version/s: 0.9.3
• Component/s: C++ - Library

Environment

Thrift running in Linux, C++

Docs Text

void callbackThreadFunction(boost::shared_ptr<TProtocol> proto)
    {
        generated::LicensingNotificationsProcessor processor(_notificationHandler);
        while (isConnected())
        {
            try
            {
                processor.process(proto, proto, NULL);
            }
            catch (const apache::thrift::transport::TTransportException& e)
            {
                if (e.getType() != apache::thrift::transport::TTransportException::TIMED_OUT)
                {
                    TRACE_DEBUG_THRIFTAPI("TTransportException: %s\n", e.what());
                    break;
                } else {
                    // FIXME take care of THRIFT_EAGAIN
                    TRACE_DEBUG_THRIFTAPI("TTransportException %s\n", e.what());
                }
            }
            catch (const std::exception& e)
            {
                TRACE_DEBUG_THRIFTAPI("exception %s\n", e.what());
                break;
            }
        }

        closeConnections();
        TRACE_DEBUG_THRIFTAPI("exit %s\n", common::Thread::id().c_str());
    }

• Assignee: James E. King III
• Reporter: leyu zhu
• Created: 04/Dec/17 15:02
• Updated: 11/Jan/19 23:02
• Resolved: 11/Jan/19 23:02

Description

In my project, the thrift version 0.9.3 is used.

In the thrift client side, the  processor.process(proto, proto, NULL) is called to handle the callback event from server side in one separate thread. (Complete function is below in Docs text)

From another thread, when the application close, the proto->getTransport()->close() is called to terminate the above processor.process(proto, proto, NULL).

Without TLS enabled, the program running very stable.

But if TLS enabled between thrift server and client, at the end of the running, about 1/2000 possibility, the program could get segement error-"Program received signal SIGSEGV, Segmentation fault.", the call stack from GDB is below.

From the call stack, can seehe error is from the method uint32_t TSSLSocket::read(uint8_t* buf, uint32_t len), in which the  "bytes = SSL_read(ssl_, buf, len) " trigger the Segmentation fault, seems the ssl_ could be released during the executing of SSL_read.

Any friends have seen the similar issue or Is TSSLSocket is thred-safe in thift version 0.9.3?

======================GDB trace================================
(gdb) bt full#0  0x00007fffe40a41f0 in ?? ()No symbol table info available.#1  0x00007ffff6d298ff in BIO_read () from /app/openssl/1.0.1e/LMWP3/lib/libcrypto.so.1.0.0No symbol table info available.#2  0x00007ffff704ba16 in ssl3_read_n () from /app/openssl/1.0.1e/LMWP3/lib/libssl.so.1.0.0No symbol table info available.#3  0x00007ffff704bf6c in ssl3_read_bytes () from /app/openssl/1.0.1e/LMWP3/lib/libssl.so.1.0.0No symbol table info available.#4  0x00007ffff704915a in ssl3_read () from /app/openssl/1.0.1e/LMWP3/lib/libssl.so.1.0.0No symbol table info available.#5  0x00007ffff6054975 in apache::thrift::transport::TSSLSocket::read (this=0x7fffe408c990, buf=0x7fffe408ced0 "", len=512) at src/thrift/transport/TSSLSocket.cpp:336        errno_copy = <optimized out>        error = <optimized out>        errors = {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> =

{<No data fields>}, <No data fields>}, _M_p = 0x0}}        retries = 1        bytes = 0#6  0x00007ffff605ad54 in read (len=<optimized out>, buf=<optimized out>, this=<optimized out>) at ./src/thrift/transport/TTransport.h:105No locals.#7  apache::thrift::transport::TBufferedTransport::readSlow (this=0x7fffe40885c0, buf=0x7ffff1b75cd0 "", len=4) at src/thrift/transport/TBufferTransports.cpp:59        have = 0        _PRETTY_FUNCTION_ = "virtual uint32_t apache::thrift::transport::TBufferedTransport::readSlow(uint8_t*, uint32_t)"        SSL_READ_RETURN = 4294967295        reading_bytes = <optimized out>#8  0x000000000049b7cd in read (len=4, buf=0x7ffff1b75cd0 "", this=0x7fffe40885c0) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TBufferTransports.h:71        new_rBase = 0x7fffe408ced5 "tionalModeizeNotificationChannel"#9  apache::thrift::transport::readAll<apache::thrift::transport::TBufferBase> (trans=..., buf=0x7ffff1b75cd0 "", len=len@entry=4)    at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TTransport.h:41        have = 0        get = <optimized out>#10 0x000000000049b8ac in readAll (len=4, buf=<optimized out>, this=0x7fffe40885c0) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TBufferTransports.h:84        new_rBase = 0x7fffe408ced5 "tionalModeizeNotificationChannel"#11 readAll (len=4, buf=<optimized out>, this=0x7fffe40885c0) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TBufferTransports.h:264No locals.#12 apache::thrift::transport::TVirtualTransport<apache::thrift::transport::TBufferedTransport, apache::thrift::transport::TBufferBase>::readAll_virt (this=0x7fffe40885c0, buf=<optimized out>, len=4)    at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TVirtualTransport.h:92No locals.#13 0x00000000004c28dc in readAll (len=4, buf=0x7ffff1b75cd0 "", this=<optimized out>) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/transport/TTransport.h:121No locals.#14 readI32 (i32=<synthetic pointer>, this=0x7fffe408c6b0) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/protocol/TBinaryProtocol.tcc:371        theBytes = {b = "\000\000\000", all = 0}#15 apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport, apache::thrift::protocol::TNetworkBigEndian>::readMessageBegin (this=0x7fffe408c6b0, name=...,     messageType=@0x7ffff1b75d58: apache::thrift::protocol::T_ONEWAY, seqid=@0x7ffff1b75d5c: 0) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/protocol/TBinaryProtocol.tcc:203        result = 0        sz = <optimized out>#16 0x00000000004c2ac1 in apache::thrift::protocol::TVirtualProtocol<apache::thrift::protocol::TBinaryProtocolT<apache::thrift::transport::TTransport, apache::thrift::protocol::TNetworkBigEndian>, apache::thrift::protocol::TProtocolDefaults>::readMessageBegin_virt (this=<optimized out>, name=..., messageType=<optimized out>, seqid=<optimized out>)    at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/protocol/TVirtualProtocol.h:403No locals.#17 0x00000000004ab0ee in readMessageBegin (seqid=@0x7ffff1b75d5c: 0, messageType=@0x7ffff1b75d58: apache::thrift::protocol::T_ONEWAY, name=..., this=0x7fffe408c6b0)    at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/protocol/TProtocol.h:431No locals.#18 process (connectionContext=0x0, out=<error reading variable: access outside bounds of object referenced via synthetic pointer>, in=<error reading variable: access outside bounds of object referenced via synthetic pointer>,     this=0x7ffff1b75d80) at /home/eleyzhu/repo/lm/3pp/thrift-0.9.3.native-x86_64_Debug/install/usr/local/include/thrift/TDispatchProcessor.h:114        fname = {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}

, <No data fields>}, _M_p = 0x971d78 ""}}        mtype = apache::thrift::protocol::T_ONEWAY        seqid = 0#19 lmclient::ThriftClient<com::ericsson::licensing::lm::LicensingControlExtendedIf>::callbackThreadFunction (this=0x7fffffffb1b0, proto=...) at /home/eleyzhu/repo/lm/src/lmServer/src/thriftAPI/fet/ThriftClient.h:1005        _func_ = "callbackThreadFunction"        processor = {<apache::thrift::TDispatchProcessor> = {<apache::thrift::TProcessor> = {vptr.TProcessor = 0x969030 <vtable for com::ericsson::licensing::lm::LicensingNotificationsProcessor+16>, eventHandler = {px = 0x0, pn =

{                  pi_ = 0x0}

}}, <No data fields>}, iface_ = {px = 0x7fffe4081e70, pn = {pi_ = 0x7fffe4081e50}}, processMap_ = {_M_t = {              _M_impl = {<std::allocator<std::Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void (com::ericsson::licensing::lm::LicensingNotificationsProcessor::)(int, apache::thri--Type <return> to continue, or q <return> to quit--ft::protocol::TProtocol, apache::thrift::protocol::TProtocol*, void*)> > >> = {<_gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> > const, void (com::ericsson::licensing::lm::LicensingNotificationsProcessor::)(int, apache::thrift::protocol::TProtocol, apache::thrift::protocol::TProtocol*, void*)> > >> =

{<No data fields>}

, <No data fields>},                 _M_key_compare = {<std::binary_function<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::all