[TS-2205] AIO caused system hang

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 4.0.1
• Component/s: Cache
• Assignee: Unassigned
• Reporter: Zhao Yongming
• Created: 10/Sep/13 09:28
• Updated: 11/Jun/15 20:56
• Resolved: 11/Jun/15 20:56

Description

the system may hang with AIO thread CPU usage rising:

top - 17:10:46 up 38 days, 22:43,  2 users,  load average: 11.34, 2.97, 2.75
Tasks: 512 total,  55 running, 457 sleeping,   0 stopped,   0 zombie
Cpu(s):  6.9%us, 54.8%sy,  0.0%ni, 37.3%id,  0.0%wa,  0.0%hi,  0.9%si,  0.0%st
Mem:  65963696k total, 64318444k used,  1645252k free,   241496k buffers
Swap: 33554424k total,    20416k used, 33534008k free, 14864188k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
32498 ats       20   0 59.3g  45g  25m R 65.8 72.1  24:44.15 [ET_AIO 5]
 3213 root      20   0     0    0    0 S 15.4  0.0  13:38.32 kondemand/7
 3219 root      20   0     0    0    0 S 15.1  0.0  16:32.78 kondemand/13
    4 root      20   0     0    0    0 S 13.8  0.0  33:18.13 ksoftirqd/0
   13 root      20   0     0    0    0 S 13.4  0.0  21:45.18 ksoftirqd/2
   37 root      20   0     0    0    0 S 13.4  0.0  19:42.34 ksoftirqd/8
   45 root      20   0     0    0    0 S 13.4  0.0  18:31.17 ksoftirqd/10
32483 ats       20   0 59.3g  45g  25m R 13.4 72.1  16:47.14 [ET_AIO 6]
32487 ats       20   0 59.3g  45g  25m R 13.4 72.1  16:46.93 [ET_AIO 2]
   25 root      20   0     0    0    0 S 13.1  0.0  19:02.18 ksoftirqd/5
   65 root      20   0     0    0    0 S 13.1  0.0  19:24.04 ksoftirqd/15
32477 ats       20   0 59.3g  45g  25m R 13.1 72.1  16:32.90 [ET_AIO 0]
32478 ats       20   0 59.3g  45g  25m R 13.1 72.1  16:49.77 [ET_AIO 1]
32479 ats       20   0 59.3g  45g  25m S 13.1 72.1  16:41.77 [ET_AIO 2]
32481 ats       20   0 59.3g  45g  25m R 13.1 72.1  16:50.40 [ET_AIO 4]
32482 ats       20   0 59.3g  45g  25m R 13.1 72.1  16:47.42 [ET_AIO 5]
32484 ats       20   0 59.3g  45g  25m R 13.1 72.1  16:25.81 [ET_AIO 7]
32485 ats       20   0 59.3g  45g  25m S 13.1 72.1  16:52.71 [ET_AIO 0]
32486 ats       20   0 59.3g  45g  25m S 13.1 72.1  16:51.69 [ET_AIO 1]
32491 ats       20   0 59.3g  45g  25m S 13.1 72.1  16:50.58 [ET_AIO 6]
32492 ats       20   0 59.3g  45g  25m S 13.1 72.1  16:49.12 [ET_AIO 7]
32480 ats       20   0 59.3g  45g  25m S 12.8 72.1  16:47.39 [ET_AIO 3]
32488 ats       20   0 59.3g  45g  25m R 12.8 72.1  16:52.16 [ET_AIO 3]
32489 ats       20   0 59.3g  45g  25m S 12.8 72.1  16:50.79 [ET_AIO 4]
32490 ats       20   0 59.3g  45g  25m R 12.8 72.1  16:52.61 [ET_AIO 5]

Issue Links

• duplicates: TS-2027 there`s no initializer of ConnectionCount`s mutex
• duplicates: TS-3458 AIO caused system hang again