2015-12-14 10:08:12,608 INFO ContainerDumpExecutor.dumpam: ************************************************************ 2015-12-14 10:08:12,609 INFO ContainerDumpExecutor.dumpam: dump AppMaster(pid=35291) information: 2015-12-14 10:08:13,130 INFO ContainerDumpExecutor.dumpam: dump JPS: 28944 YarnChild 38808 YarnChild 38473 YarnChild 36934 YarnChild 48822 MRAppMaster 38266 YarnChild 39531 Jps 38075 YarnChild 39289 YarnChild 43557 YarnChild 18312 NodeManager 37353 YarnChild 35291 MRAppMaster 35914 YarnChild 39145 YarnChild 2015-12-14 10:08:13,130 INFO ContainerDumpExecutor.dumpam: dump ps master: yarn 35291 47.8 0.5 1771580 388764 ? Sl 10:07 0:13 /software/servers/jdk1.7.0_67/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/data11/yarn1/logs/application_1449835724839_178210/container_1449835724839_178210_01_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 2015-12-14 10:08:14,765 INFO ContainerDumpExecutor.dumpam: dump pstack: Thread 92 (Thread 0x7f8f282a9700 (LWP 35292)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b0fd in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f28c02bb1 in Threads::destroy_vm() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288e25f0 in jni_DestroyJavaVM () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f2912416d in JavaMain () from /software/servers/jdk1.7.0_67/bin/../lib/amd64/jli/libjli.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 91 (Thread 0x7f8f2062b700 (LWP 35293)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 90 (Thread 0x7f8f2052a700 (LWP 35294)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 89 (Thread 0x7f8f20429700 (LWP 35295)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 88 (Thread 0x7f8f20328700 (LWP 35296)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 87 (Thread 0x7f8f20227700 (LWP 35297)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 86 (Thread 0x7f8f20126700 (LWP 35298)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 85 (Thread 0x7f8f20025700 (LWP 35299)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 84 (Thread 0x7f8f1ff24700 (LWP 35300)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 83 (Thread 0x7f8f1fe23700 (LWP 35301)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 82 (Thread 0x7f8f1fd22700 (LWP 35302)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 81 (Thread 0x7f8f1fc21700 (LWP 35303)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 80 (Thread 0x7f8f1fb20700 (LWP 35304)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 79 (Thread 0x7f8f1fa1f700 (LWP 35305)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 78 (Thread 0x7f8f1f91e700 (LWP 35306)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 77 (Thread 0x7f8f1f81d700 (LWP 35307)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 76 (Thread 0x7f8f1f71c700 (LWP 35308)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 75 (Thread 0x7f8f1f61b700 (LWP 35309)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 74 (Thread 0x7f8f1f51a700 (LWP 35310)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 73 (Thread 0x7f8f1f419700 (LWP 35311)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 72 (Thread 0x7f8f1f318700 (LWP 35312)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 71 (Thread 0x7f8f1f217700 (LWP 35313)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 70 (Thread 0x7f8f1f116700 (LWP 35314)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 69 (Thread 0x7f8f1f015700 (LWP 35315)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f287ff9e3 in GCTaskManager::get_task(unsigned int) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f288010a8 in GCTaskThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 68 (Thread 0x7f8f0fafa700 (LWP 35316)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8ad5e in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f28c50169 in VMThread::loop() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f28c50470 in VMThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 67 (Thread 0x7f8f0f9f9700 (LWP 35317)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab7add in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f0f9f8990 in ?? () #6 0x00007f8f2409afc0 in ?? () #7 0x00007f8f0f9f87c0 in ?? () #8 0x00007f8f0f9f8758 in ?? () #9 0x0000000000000000 in ?? () Thread 66 (Thread 0x7f8f0f8f8700 (LWP 35318)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab7add in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f0f8f77e0 in ?? () #6 0x00007f8f2409f800 in ?? () #7 0x00007f8f240a02a0 in ?? () #8 0x00007f8f0f8f7768 in ?? () #9 0x0000000000000000 in ?? () Thread 65 (Thread 0x7f8f09967700 (LWP 35320)): #0 0x0000003c48c0d720 in sem_wait () from /lib64/libpthread.so.0 #1 0x00007f8f28ac85ca in check_pending_signals(bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ac1835 in signal_thread_entry(JavaThread*, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28c02d9f in JavaThread::thread_main_inner() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f28c02ea5 in JavaThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #7 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 64 (Thread 0x7f8f09866700 (LWP 35321)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b2b6 in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f286f4ad8 in CompileQueue::get() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f286f8bda in CompileBroker::compiler_thread_loop() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28c02d9f in JavaThread::thread_main_inner() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f28c02ea5 in JavaThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #8 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #9 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #10 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 63 (Thread 0x7f8f09765700 (LWP 35322)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b2b6 in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f286f4ad8 in CompileQueue::get() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f286f8bda in CompileBroker::compiler_thread_loop() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28c02d9f in JavaThread::thread_main_inner() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f28c02ea5 in JavaThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #8 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #9 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #10 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 62 (Thread 0x7f8f09664700 (LWP 35323)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8aaaf in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f28b59388 in ServiceThread::service_thread_entry(JavaThread*, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f28c02d9f in JavaThread::thread_main_inner() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28c02ea5 in JavaThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #8 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #9 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 61 (Thread 0x7f8f09563700 (LWP 35324)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28a8ad5e in Monitor::IWait(Thread*, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28a8b23e in Monitor::wait(bool, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f28bfedc1 in WatcherThread::sleep() const () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f28bff2de in WatcherThread::run() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #6 0x00007f8f28ac9988 in java_start(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x0000003c48c07851 in start_thread () from /lib64/libpthread.so.0 #8 0x0000003c484e811d in clone () from /lib64/libc.so.6 Thread 60 (Thread 0x7f8f083e9700 (LWP 35340)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab4e1e in ObjectMonitor::EnterI(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28ab6c79 in ObjectMonitor::enter(Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f288a2d71 in InterpreterRuntime::monitorenter(JavaThread*, BasicObjectLock*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #5 0x00007f8f209bc972 in ?? () #6 0x00007f8f209bc930 in ?? () #7 0x0000000000000003 in ?? () #8 0x00000000ebde4e00 in ?? () #9 0x00007f8f083e8588 in ?? () #10 0x00000000bb7d0c78 in ?? () #11 0x00007f8f083e85f8 in ?? () #12 0x00000000bb7d1138 in ?? () #13 0x0000000000000000 in ?? () Thread 59 (Thread 0x7f8f07e9a700 (LWP 35618)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8114 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x0000000000000001 in ?? () #5 0x00007f8f255bc000 in ?? () #6 0x0000000000000000 in ?? () Thread 58 (Thread 0x7f8f07d99700 (LWP 35626)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab737b in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f07d98880 in ?? () #6 0x00007f8f28ac3123 in os::javaTimeMillis() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f209b2058 in ?? () #8 0x0000000000000000 in ?? () Thread 57 (Thread 0x7f8f07c98700 (LWP 35640)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ac93d4 in os::sleep(Thread*, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f2892f892 in JVM_Sleep () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f257f3000 in ?? () #6 0x00007f8f209b2175 in ?? () #7 0x00007f8f07c974b0 in ?? () #8 0x00000000bae327e8 in ?? () #9 0x00007f8f07c97518 in ?? () #10 0x00000000baeb2988 in ?? () #11 0x0000000000000000 in ?? () Thread 56 (Thread 0x7f8f07b97700 (LWP 35678)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48789 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x00000000bae34118 in ?? () #4 0x0000000000000000 in ?? () Thread 55 (Thread 0x7f8f07a96700 (LWP 35691)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48707 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x0000000000000000 in ?? () Thread 54 (Thread 0x7f8f07995700 (LWP 35692)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48789 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x00000000bae34118 in ?? () #4 0x0000000000000000 in ?? () Thread 53 (Thread 0x7f8f07894700 (LWP 35693)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 52 (Thread 0x7f8f07793700 (LWP 35728)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab737b in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00000000017e36d8 in ?? () #6 0x00000000000000b6 in ?? () #7 0x00007f8f258de800 in ?? () #8 0x00007f8f20a09a47 in ?? () #9 0x00007f8f077927e0 in ?? () #10 0x0000000000000000 in ?? () Thread 51 (Thread 0x7f8f07692700 (LWP 35729)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48707 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x00000000bae34118 in ?? () #4 0x0000000000000000 in ?? () Thread 50 (Thread 0x7f8f07591700 (LWP 35850)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab737b in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00000000bae027a0 in ?? () #6 0x00007f8f28ac3123 in os::javaTimeMillis() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f209b2058 in ?? () #8 0x0000000000000000 in ?? () Thread 49 (Thread 0x7f8f07490700 (LWP 35908)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab7add in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f209b1310 in ?? () #6 0x0000000000000000 in ?? () Thread 48 (Thread 0x7f8f0738f700 (LWP 35955)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab7add in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x0000000000000001 in ?? () #6 0x0000000000000000 in ?? () Thread 47 (Thread 0x7f8f0728e700 (LWP 35956)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 46 (Thread 0x7f8f0718d700 (LWP 35957)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48789 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x006e006900740072 in ?? () #4 0x00007f8f209bf2ac in ?? () #5 0x0000000000000000 in ?? () Thread 45 (Thread 0x7f8f0708c700 (LWP 35967)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48707 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x0000000000000000 in ?? () Thread 44 (Thread 0x7f8f06f8b700 (LWP 35968)): #0 0x0000003c484e8713 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f8f08e48789 in Java_sun_nio_ch_EPollArrayWrapper_epollWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libnio.so #2 0x00007f8f209bed98 in ?? () #3 0x00000000bafe7628 in ?? () #4 0x0000000000000000 in ?? () Thread 43 (Thread 0x7f8f06e8a700 (LWP 35969)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 42 (Thread 0x7f8f06d89700 (LWP 35970)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 41 (Thread 0x7f8f06c88700 (LWP 35971)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 40 (Thread 0x7f8f06b87700 (LWP 35972)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 39 (Thread 0x7f8f06a86700 (LWP 35973)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 38 (Thread 0x7f8f06985700 (LWP 35974)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 37 (Thread 0x7f8f06884700 (LWP 35975)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 36 (Thread 0x7f8f06783700 (LWP 35976)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 35 (Thread 0x7f8f06682700 (LWP 35977)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 34 (Thread 0x7f8f06581700 (LWP 35978)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 33 (Thread 0x7f8f06480700 (LWP 35979)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 32 (Thread 0x7f8f0637f700 (LWP 35980)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 31 (Thread 0x7f8f0627e700 (LWP 35981)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 30 (Thread 0x7f8f0617d700 (LWP 35982)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 29 (Thread 0x7f8f0607c700 (LWP 35983)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 28 (Thread 0x7f8f05f7b700 (LWP 35984)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 27 (Thread 0x7f8f05e7a700 (LWP 35985)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 26 (Thread 0x7f8f05d79700 (LWP 35986)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 25 (Thread 0x7f8f05c78700 (LWP 35987)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 24 (Thread 0x7f8f05b77700 (LWP 35988)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 23 (Thread 0x7f8f05a76700 (LWP 35989)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 22 (Thread 0x7f8f05975700 (LWP 35990)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 21 (Thread 0x7f8f05874700 (LWP 35991)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 20 (Thread 0x7f8f05773700 (LWP 35992)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 19 (Thread 0x7f8f05672700 (LWP 35993)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 18 (Thread 0x7f8f05571700 (LWP 35994)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 17 (Thread 0x7f8f05470700 (LWP 35995)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 16 (Thread 0x7f8f0536f700 (LWP 35996)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 15 (Thread 0x7f8f0526e700 (LWP 35997)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 14 (Thread 0x7f8f0516d700 (LWP 35998)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 13 (Thread 0x7f8f0506c700 (LWP 35999)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ac93d4 in os::sleep(Thread*, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f2892f892 in JVM_Sleep () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f25bc0c98 in ?? () #6 0x00007f8f0506b780 in ?? () #7 0x00007f8f25bc0000 in ?? () #8 0x00007f8f0506b768 in ?? () #9 0x0000000000000000 in ?? () Thread 12 (Thread 0x7f8f04f6b700 (LWP 36000)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8114 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 11 (Thread 0x7f8f04e6a700 (LWP 36001)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f04e697d8 in ?? () #5 0x00007f8f209b2058 in ?? () #6 0x0000000000000000 in ?? () Thread 10 (Thread 0x7f8f04d69700 (LWP 36002)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab737b in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x0000000000000000 in ?? () Thread 9 (Thread 0x7f8f04b67700 (LWP 36004)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f04b66538 in ?? () #5 0x00007f8f209b2058 in ?? () #6 0x0000000000000000 in ?? () Thread 8 (Thread 0x7f8f04a66700 (LWP 36005)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 7 (Thread 0x7f8f04965700 (LWP 36006)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ac93d4 in os::sleep(Thread*, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f2892f892 in JVM_Sleep () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f259b9800 in ?? () #6 0x000001519e3e5d86 in ?? () #7 0x00007f8f049646e0 in ?? () #8 0x0000000000000000 in ?? () Thread 6 (Thread 0x7f8f04864700 (LWP 36007)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac8228 in Parker::park(bool, long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28c26a05 in Unsafe_Park () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 5 (Thread 0x7f8f04763700 (LWP 36008)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab737b in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00000000baeb1428 in ?? () #6 0x0000000000000000 in ?? () Thread 4 (Thread 0x7f8f04662700 (LWP 36009)): #0 0x0000003c48c0b7bb in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac88ef in os::PlatformEvent::park(long) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ac93d4 in os::sleep(Thread*, long, bool) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f2892f892 in JVM_Sleep () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f209b29fe in ?? () #6 0x00000000000003e8 in ?? () #7 0x00007f8f04661750 in ?? () #8 0x0000000000000000 in ?? () Thread 3 (Thread 0x7f8f04460700 (LWP 36074)): #0 0x0000003c48c0b43c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f8f28ac3243 in os::PlatformEvent::park() () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f28ab7add in ObjectMonitor::wait(long, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #3 0x00007f8f28927378 in JVM_MonitorWait () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #4 0x00007f8f209bed98 in ?? () #5 0x00007f8f0445f970 in ?? () #6 0x00007f8f289edead in LinkResolver::resolve_method(methodHandle&, KlassHandle, Symbol*, Symbol*, KlassHandle, bool, Thread*) () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #7 0x00007f8f209b2058 in ?? () #8 0x0000000000000000 in ?? () Thread 2 (Thread 0x7f8f083b0700 (LWP 39720)): #0 0x0000003c48c0e54d in read () from /lib64/libpthread.so.0 #1 0x00007f8f289207ae in JVM_Read () from /software/servers/jdk1.7.0_67/jre/lib/amd64/server/libjvm.so #2 0x00007f8f23bd9c5e in readBytes () from /software/servers/jdk1.7.0_67/jre/lib/amd64/libjava.so #3 0x00007f8f209bed98 in ?? () #4 0x00007f8f209b2058 in ?? () #5 0x0000000000000000 in ?? () Thread 1 (Thread 0x7f8f2911f700 (LWP 35291)): #0 0x0000003c48c080ad in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f8f2912e505 in ContinueInNewThread0 () from /software/servers/jdk1.7.0_67/bin/../lib/amd64/jli/libjli.so #2 0x00007f8f2912358a in ContinueInNewThread () from /software/servers/jdk1.7.0_67/bin/../lib/amd64/jli/libjli.so #3 0x00007f8f291260e0 in JLI_Launch () from /software/servers/jdk1.7.0_67/bin/../lib/amd64/jli/libjli.so #4 0x0000000000400686 in main () 2015-12-14 10:08:17,098 INFO ContainerDumpExecutor.dumpam: dump jstat: S0 S1 E O P YGC YGCT FGC FGCT GCT 0.00 24.70 9.93 0.01 98.72 3 0.082 0 0.000 0.082 0.00 24.70 16.30 0.01 98.72 3 0.082 0 0.000 0.082 0.00 24.70 22.24 0.01 98.72 3 0.082 0 0.000 0.082 2015-12-14 10:08:17,666 INFO ContainerDumpExecutor.dumpam: dump jstack: 2015-12-14 10:08:17 Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode): "Thread-1120" prio=10 tid=0x00000000018da000 nid=0x9c6e runnable [0x00007f8f083af000] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:272) at java.io.BufferedInputStream.read1(BufferedInputStream.java:273) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked <0x00000000eef8a5f8> (a java.lang.UNIXProcess$ProcessPipeInputStream) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked <0x00000000eef8c720> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:154) at java.io.BufferedReader.readLine(BufferedReader.java:317) - locked <0x00000000eef8c720> (a java.io.InputStreamReader) at java.io.BufferedReader.readLine(BufferedReader.java:382) at org.apache.hadoop.util.Shell$1.run(Shell.java:462) "Attach Listener" daemon prio=10 tid=0x00007f8f1000c000 nid=0x9c5b waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "FlushTimer" daemon prio=10 tid=0x00007f8f1803e800 nid=0x8cea in Object.wait() [0x00007f8f0445f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa718070> (a java.util.TaskQueue) at java.lang.Object.wait(Object.java:503) at java.util.TimerThread.mainLoop(Timer.java:526) - locked <0x00000000fa718070> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) "LeaseRenewer:user1_read@ns1" daemon prio=10 tid=0x00007f8f18016000 nid=0x8ca9 waiting on condition [0x00007f8f04661000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:438) at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71) at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:298) at java.lang.Thread.run(Thread.java:745) "Thread-53" daemon prio=10 tid=0x00007f8f18013000 nid=0x8ca8 in Object.wait() [0x00007f8f04762000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa718140> (a java.util.LinkedList) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:491) - locked <0x00000000fa718140> (a java.util.LinkedList) "CommitterEvent Processor #0" prio=10 tid=0x00007f8f18007800 nid=0x8ca7 waiting on condition [0x00007f8f04863000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa7181c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "DestroyJavaVM" prio=10 tid=0x00007f8f24010000 nid=0x89dc waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Thread-52" prio=10 tid=0x00007f8f259b9800 nid=0x8ca6 waiting on condition [0x00007f8f04964000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1064) "eventHandlingThread" prio=10 tid=0x00007f8f259b5800 nid=0x8ca5 waiting on condition [0x00007f8f04a65000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa738068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:263) at java.lang.Thread.run(Thread.java:745) "ContainerLauncher Event Handler" prio=10 tid=0x00007f8f259b3000 nid=0x8ca4 waiting on condition [0x00007f8f04b66000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa7280d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$1.run(ContainerLauncherImpl.java:271) "IPC Client (1695611352) connection to rmhost:8030 from user1_read" daemon prio=10 tid=0x00007f8f25f08000 nid=0x8ca2 in Object.wait() [0x00007f8f04d68000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa740000> (a org.apache.hadoop.ipc.Client$Connection) at org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:845) - locked <0x00000000fa740000> (a org.apache.hadoop.ipc.Client$Connection) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:890) "Thread-48" prio=10 tid=0x00007f8f25d78800 nid=0x8ca1 waiting on condition [0x00007f8f04e69000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa738190> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator$1.run(RMContainerAllocator.java:191) "DefaultSpeculator background processing" prio=10 tid=0x00007f8f25bc3000 nid=0x8ca0 waiting on condition [0x00007f8f04f6a000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa7281d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) at org.apache.hadoop.mapreduce.v2.app.speculate.DefaultSpeculator$1.run(DefaultSpeculator.java:200) at java.lang.Thread.run(Thread.java:745) "TaskHeartbeatHandler PingChecker" prio=10 tid=0x00007f8f25bc0000 nid=0x8c9f waiting on condition [0x00007f8f0506b000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler$PingChecker.run(TaskHeartbeatHandler.java:157) at java.lang.Thread.run(Thread.java:745) "IPC Server handler 29 on 61296" daemon prio=10 tid=0x00007f8f25bbd800 nid=0x8c9e waiting on condition [0x00007f8f0516c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 28 on 61296" daemon prio=10 tid=0x00007f8f25d88800 nid=0x8c9d waiting on condition [0x00007f8f0526d000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 27 on 61296" daemon prio=10 tid=0x00007f8f25d86800 nid=0x8c9c waiting on condition [0x00007f8f0536e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 26 on 61296" daemon prio=10 tid=0x00007f8f25d84000 nid=0x8c9b waiting on condition [0x00007f8f0546f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 25 on 61296" daemon prio=10 tid=0x00007f8f25d82000 nid=0x8c9a waiting on condition [0x00007f8f05570000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 24 on 61296" daemon prio=10 tid=0x00007f8f25d96800 nid=0x8c99 waiting on condition [0x00007f8f05671000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 23 on 61296" daemon prio=10 tid=0x00007f8f25d94800 nid=0x8c98 waiting on condition [0x00007f8f05772000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 22 on 61296" daemon prio=10 tid=0x00007f8f25d92800 nid=0x8c97 waiting on condition [0x00007f8f05873000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 21 on 61296" daemon prio=10 tid=0x00007f8f25d90800 nid=0x8c96 waiting on condition [0x00007f8f05974000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 20 on 61296" daemon prio=10 tid=0x00007f8f25d8e000 nid=0x8c95 waiting on condition [0x00007f8f05a75000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 19 on 61296" daemon prio=10 tid=0x00007f8f25d8c000 nid=0x8c94 waiting on condition [0x00007f8f05b76000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 18 on 61296" daemon prio=10 tid=0x00007f8f25d8a000 nid=0x8c93 waiting on condition [0x00007f8f05c77000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 17 on 61296" daemon prio=10 tid=0x00007f8f25d80000 nid=0x8c92 waiting on condition [0x00007f8f05d78000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 16 on 61296" daemon prio=10 tid=0x00007f8f25d7e000 nid=0x8c91 waiting on condition [0x00007f8f05e79000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 15 on 61296" daemon prio=10 tid=0x00007f8f25d7b800 nid=0x8c90 waiting on condition [0x00007f8f05f7a000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 14 on 61296" daemon prio=10 tid=0x00007f8f259ce000 nid=0x8c8f waiting on condition [0x00007f8f0607b000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 13 on 61296" daemon prio=10 tid=0x00007f8f259cc000 nid=0x8c8e waiting on condition [0x00007f8f0617c000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 12 on 61296" daemon prio=10 tid=0x00007f8f259c9800 nid=0x8c8d waiting on condition [0x00007f8f0627d000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 11 on 61296" daemon prio=10 tid=0x00007f8f259c7800 nid=0x8c8c waiting on condition [0x00007f8f0637e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 10 on 61296" daemon prio=10 tid=0x00007f8f259c6000 nid=0x8c8b waiting on condition [0x00007f8f0647f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 9 on 61296" daemon prio=10 tid=0x00007f8f25bdd000 nid=0x8c8a waiting on condition [0x00007f8f06580000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 8 on 61296" daemon prio=10 tid=0x00007f8f25bdb000 nid=0x8c89 waiting on condition [0x00007f8f06681000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 7 on 61296" daemon prio=10 tid=0x00007f8f25bd9000 nid=0x8c88 waiting on condition [0x00007f8f06782000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 6 on 61296" daemon prio=10 tid=0x00007f8f25a2c000 nid=0x8c87 waiting on condition [0x00007f8f06883000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 5 on 61296" daemon prio=10 tid=0x00007f8f25a2a000 nid=0x8c86 waiting on condition [0x00007f8f06984000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 4 on 61296" daemon prio=10 tid=0x00007f8f25a28000 nid=0x8c85 waiting on condition [0x00007f8f06a85000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 3 on 61296" daemon prio=10 tid=0x00007f8f25a93800 nid=0x8c84 waiting on condition [0x00007f8f06b86000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 2 on 61296" daemon prio=10 tid=0x00007f8f25a91800 nid=0x8c83 waiting on condition [0x00007f8f06c87000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 1 on 61296" daemon prio=10 tid=0x00007f8f25a8f800 nid=0x8c82 waiting on condition [0x00007f8f06d88000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server handler 0 on 61296" daemon prio=10 tid=0x00007f8f25a03000 nid=0x8c81 waiting on condition [0x00007f8f06e89000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa720138> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server listener on 61296" daemon prio=10 tid=0x00007f8f25c19800 nid=0x8c80 runnable [0x00007f8f06f8a000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa750470> (a sun.nio.ch.Util$2) - locked <0x00000000fa750480> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa750428> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) at org.apache.hadoop.ipc.Server$Listener.run(Server.java:693) "IPC Server Responder" daemon prio=10 tid=0x00007f8f25c18000 nid=0x8c7f runnable [0x00007f8f0708b000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa758258> (a sun.nio.ch.Util$2) - locked <0x00000000fa758268> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa758210> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:881) at org.apache.hadoop.ipc.Server$Responder.run(Server.java:864) "Socket Reader #1 for port 61296" prio=10 tid=0x00007f8f259fe800 nid=0x8c75 runnable [0x00007f8f0718c000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa7205d8> (a sun.nio.ch.Util$2) - locked <0x00000000fa7205e8> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa720590> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:580) - locked <0x00000000fa7204d0> (a org.apache.hadoop.ipc.Server$Listener$Reader) at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:566) "CommitterEvent Handler" prio=10 tid=0x00007f8f25c0e800 nid=0x8c74 waiting on condition [0x00007f8f0728d000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa748500> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$1.run(CommitterEventHandler.java:123) at java.lang.Thread.run(Thread.java:745) "AsyncDispatcher event handler" prio=10 tid=0x00007f8f25c0d800 nid=0x8c73 runnable [0x00007f8f0738d000] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:272) at java.io.BufferedInputStream.read1(BufferedInputStream.java:273) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked <0x00000000eef88530> (a java.lang.UNIXProcess$ProcessPipeInputStream) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177) - locked <0x00000000eef92810> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:154) at java.io.BufferedReader.read1(BufferedReader.java:205) at java.io.BufferedReader.read(BufferedReader.java:279) - locked <0x00000000eef92810> (a java.io.InputStreamReader) at org.apache.hadoop.util.Shell$ShellCommandExecutor.parseExecResult(Shell.java:633) at org.apache.hadoop.util.Shell.runCommand(Shell.java:477) at org.apache.hadoop.util.Shell.run(Shell.java:410) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:620) at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.runResolveCommand(ScriptBasedMapping.java:241) at org.apache.hadoop.net.ScriptBasedMapping$RawScriptBasedMapping.resolve(ScriptBasedMapping.java:179) at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119) at org.apache.hadoop.yarn.util.RackResolver.coreResolve(RackResolver.java:101) at org.apache.hadoop.yarn.util.RackResolver.resolve(RackResolver.java:95) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl.(TaskAttemptImpl.java:548) at org.apache.hadoop.mapred.MapTaskAttemptImpl.(MapTaskAttemptImpl.java:47) at org.apache.hadoop.mapreduce.v2.app.job.impl.MapTaskImpl.createAttempt(MapTaskImpl.java:62) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.addAttempt(TaskImpl.java:594) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.addAndScheduleAttempt(TaskImpl.java:581) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.access$1300(TaskImpl.java:100) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl$InitialScheduleTransition.transition(TaskImpl.java:871) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl$InitialScheduleTransition.transition(TaskImpl.java:866) at org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362) at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302) at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46) at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448) - locked <0x00000000faba92f8> (a org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.handle(TaskImpl.java:632) at org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl.handle(TaskImpl.java:99) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher.handle(MRAppMaster.java:1366) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher.handle(MRAppMaster.java:1360) at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:134) at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:81) at java.lang.Thread.run(Thread.java:745) "com.google.inject.internal.util.$Finalizer" daemon prio=10 tid=0x00007f8f2594d800 nid=0x8c44 in Object.wait() [0x00007f8f0748f000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa778008> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x00000000fa778008> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at com.google.inject.internal.util.$Finalizer.run(Finalizer.java:114) "Timer-0" daemon prio=10 tid=0x00007f8f25940000 nid=0x8c0a in Object.wait() [0x00007f8f07590000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa790000> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x00000000fa790000> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) "796363646@qtp-1270827248-1 - Acceptor0 SelectChannelConnector@0.0.0.0:58979" daemon prio=10 tid=0x00007f8f258dc000 nid=0x8b91 runnable [0x00007f8f07691000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa758378> (a sun.nio.ch.Util$2) - locked <0x00000000fa758388> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa758330> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498) at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192) at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124) at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) "898610278@qtp-1270827248-0" daemon prio=10 tid=0x00007f8f258de800 nid=0x8b90 in Object.wait() [0x00007f8f07792000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa718430> (a org.mortbay.thread.QueuedThreadPool$PoolThread) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626) - locked <0x00000000fa718430> (a org.mortbay.thread.QueuedThreadPool$PoolThread) "IPC Server handler 0 on 19194" daemon prio=10 tid=0x00007f8f2586d800 nid=0x8b6d waiting on condition [0x00007f8f07893000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa7780a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2024) "IPC Server listener on 19194" daemon prio=10 tid=0x00007f8f2586c000 nid=0x8b6c runnable [0x00007f8f07994000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa750948> (a sun.nio.ch.Util$2) - locked <0x00000000fa750958> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa750900> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) at org.apache.hadoop.ipc.Server$Listener.run(Server.java:693) "IPC Server Responder" daemon prio=10 tid=0x00007f8f25823800 nid=0x8b6b runnable [0x00007f8f07a95000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa7b00a8> (a sun.nio.ch.Util$2) - locked <0x00000000fa7b00b8> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa7b0060> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:881) at org.apache.hadoop.ipc.Server$Responder.run(Server.java:864) "Socket Reader #1 for port 19194" prio=10 tid=0x00007f8f25836000 nid=0x8b5e runnable [0x00007f8f07b96000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) - locked <0x00000000fa778250> (a sun.nio.ch.Util$2) - locked <0x00000000fa778260> (a java.util.Collections$UnmodifiableSet) - locked <0x00000000fa778208> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:580) - locked <0x00000000fa778148> (a org.apache.hadoop.ipc.Server$Listener$Reader) at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:566) "org.apache.hadoop.hdfs.PeerCache@4adf1d27" daemon prio=10 tid=0x00007f8f257f3000 nid=0x8b38 waiting on condition [0x00007f8f07c97000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:252) at org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:39) at org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:135) at java.lang.Thread.run(Thread.java:745) "Timer for 'MRAppMaster' metrics system" daemon prio=10 tid=0x00007f8f25790000 nid=0x8b2a in Object.wait() [0x00007f8f07d98000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa719360> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:552) - locked <0x00000000fa719360> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:505) "IPC Parameter Sending Thread #0" daemon prio=10 tid=0x00007f8f255bc000 nid=0x8b22 waiting on condition [0x00007f8f07e99000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000000fa7d0018> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "process reaper" daemon prio=10 tid=0x00007f8f24a07800 nid=0x8a0c runnable [0x00007f8f083e8000] java.lang.Thread.State: RUNNABLE at java.lang.UNIXProcess.waitForProcessExit(Native Method) at java.lang.UNIXProcess.access$500(UNIXProcess.java:54) at java.lang.UNIXProcess$4.run(UNIXProcess.java:227) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "Service Thread" daemon prio=10 tid=0x00007f8f240ce800 nid=0x89fb runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" daemon prio=10 tid=0x00007f8f240cc000 nid=0x89fa waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" daemon prio=10 tid=0x00007f8f240c8800 nid=0x89f9 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x00007f8f240c6800 nid=0x89f8 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x00007f8f2409f800 nid=0x89f6 in Object.wait() [0x00007f8f0f8f7000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa7785e0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x00000000fa7785e0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) "Reference Handler" daemon prio=10 tid=0x00007f8f2409d800 nid=0x89f5 in Object.wait() [0x00007f8f0f9f8000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000000fa7b0268> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:503) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) - locked <0x00000000fa7b0268> (a java.lang.ref.Reference$Lock) "VM Thread" prio=10 tid=0x00007f8f24099000 nid=0x89f4 runnable "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f8f24026000 nid=0x89dd runnable "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f8f24028000 nid=0x89de runnable "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f8f24029800 nid=0x89df runnable "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f8f2402b800 nid=0x89e0 runnable "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f8f2402d800 nid=0x89e1 runnable "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f8f2402f000 nid=0x89e2 runnable "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f8f24031000 nid=0x89e3 runnable "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f8f24033000 nid=0x89e4 runnable "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007f8f24035000 nid=0x89e5 runnable "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007f8f24036800 nid=0x89e6 runnable "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007f8f24038800 nid=0x89e7 runnable "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007f8f2403a800 nid=0x89e8 runnable "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007f8f2403c000 nid=0x89e9 runnable "GC task thread#13 (ParallelGC)" prio=10 tid=0x00007f8f2403e000 nid=0x89ea runnable "GC task thread#14 (ParallelGC)" prio=10 tid=0x00007f8f24040000 nid=0x89eb runnable "GC task thread#15 (ParallelGC)" prio=10 tid=0x00007f8f24041800 nid=0x89ec runnable "GC task thread#16 (ParallelGC)" prio=10 tid=0x00007f8f24043800 nid=0x89ed runnable "GC task thread#17 (ParallelGC)" prio=10 tid=0x00007f8f24045800 nid=0x89ee runnable "GC task thread#18 (ParallelGC)" prio=10 tid=0x00007f8f24047800 nid=0x89ef runnable "GC task thread#19 (ParallelGC)" prio=10 tid=0x00007f8f24049000 nid=0x89f0 runnable "GC task thread#20 (ParallelGC)" prio=10 tid=0x00007f8f2404b000 nid=0x89f1 runnable "GC task thread#21 (ParallelGC)" prio=10 tid=0x00007f8f2404d000 nid=0x89f2 runnable "GC task thread#22 (ParallelGC)" prio=10 tid=0x00007f8f2404e800 nid=0x89f3 runnable "VM Periodic Task Thread" prio=10 tid=0x00007f8f24103000 nid=0x89fc waiting on condition JNI global references: 174