[TS-3797] Crashes due to cross-thread race conditions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.3.0
• Fix Version/s: 6.1.0
• Component/s: Core
• Labels: yahoo
• Assignee: Susan Hinrichs
• Reporter: Susan Hinrichs
• Created: 24/Jul/15 01:06
• Updated: 09/Feb/16 15:02
• Resolved: 24/Aug/15 15:53

Description

We had seen crashes with the following stack trace occasionally, but recently we have found an environment where these crashes happen so frequently that running ATS with global session pools is not feasible.

#0  0x00000000004fac6e in Ptr<IOBufferBlock>::operator IOBufferBlock* (
    this=0x10) at ../lib/ts/Ptr.h:300
#1  0x00000000005109a2 in IOBufferReader::read_avail (this=0x0)
    at ../iocore/eventsystem/P_IOBuffer.h:606
#2  0x0000000000777b54 in write_to_net_io (nh=0x2acc365358a0, 
    vc=0x2acd38024960, thread=0x2acc36532010) at UnixNetVConnection.cc:540
#3  0x000000000077747a in write_to_net (nh=0x2acc365358a0, vc=0x2acd38024960, 
    thread=0x2acc36532010) at UnixNetVConnection.cc:407
#4  0x0000000000770378 in NetHandler::mainNetEvent (this=0x2acc365358a0, 
    event=5, e=0x2244730) at UnixNet.cc:562
#5  0x0000000000510560 in Continuation::handleEvent (this=0x2acc365358a0, 
    event=5, data=0x2244730) at ../iocore/eventsystem/I_Continuation.h:145
#6  0x0000000000796ffe in EThread::process_event (this=0x2acc36532010, 
    e=0x2244730, calling_code=5) at UnixEThread.cc:128
#7  0x0000000000797508 in EThread::execute (this=0x2acc36532010)
    at UnixEThread.cc:252
#8  0x00000000007965a9 in spawn_thread_internal (a=0x2115540) at Thread.cc:85
#9  0x00002acc2edd49d1 in start_thread () from /lib64/libpthread.so.0
#10 0x00000032750e88fd in clone () from /lib64/libc.so.6

See https://cwiki.apache.org/confluence/display/TS/Threading+Issues+And+NetVC+Migration for analysis of the crash and a suggested solution.

Issue Links

• is related to: TS-4187 connections_currently_open stat not accurate with global server session pools