[TS-980] change client_session schedule from global to thread local, and reduce the try_locks in UnixNetVConnection::reenable

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 3.1.0, 3.0.0
• Component/s: Network, Performance

Environment

all

• Assignee: Wei Jin
• Reporter: weijin
• Created: 12/Oct/11 10:15
• Updated: 04/Nov/14 18:24
• Resolved: 04/Nov/14 18:24

Description

I did some performance test on ats last days(disable cache, set share_server session 2, pure proxy mode), I did see significant improvement on low load, but it dropped rapidly when load is high. meanwhile, some stability problems happened. Through gdb, I found the client_session`s mutex can be acquired by two or more threads, I believe some schedules happened during the sm life_time. May be we need do some work to find these eventProcessor.schedules and change them to thread schedules.

UnixVConnecton::reenable {
if (nh->mutex->thread_holding == t) {
// put into ready_list
} else {
MUTEX_TRY_LOCK(lock, nh->mutex, t);
if (!lock)

{ // put into enable_list; }

else

{ // put into ready_list; }

}
remove UnixNetVConnection::reenable try_lock operations, 3 reasons
1. try_lock operation means obj allocation and deallocation operation. frequently
2. try_lock hardly can lock the net-handler`s mutex.(net-handler is schedule by as soon as possible)
3. try_lock should not acquire the net-handler`s mutex. That may lead more net io latency if it is an epoll event need to be processed in other threads. If it is not an epoll event(time event), I don`t think putting vc in ready_list has any advantage than in enable_list.

may be we can change reenale function like this:
UnixVConnecton::reenable {
if (nh->mutex->thread_holding == t) {
// put into ready_list;
} else {
// put into enable_list;
}

my buddies, any advice?