Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4769

forman spins query int32 data with snappy compression

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.7.0
    • None
    • Storage - Parquet
    • None

    Description

      Similar data structure as DRILL-4767, but with SNAPPY compression, same query, drill forman just spins.

      [root@perfnode166 parquet-mr]# java -jar parquet-tools/target/parquet-tools-1.8.2-SNAPSHOT.jar dump /mapr/drill50.perf.lab/drill/testdata/parquet_storage/int32_10_bs10k_ps1k_snappy.parquet
      row group 0
      --------------------------------------------------------------------------------
      int32_field_required:  INT32 SNAPPY DO:0 FPO:4 SZ:49/47/0.96 VC:10 ENC [more]...
      int32_field_optional:  INT32 SNAPPY DO:0 FPO:53 SZ:51/49/0.96 VC:10 EN [more]...
      int32_field_repeated:  INT32 SNAPPY DO:0 FPO:104 SZ:53/51/0.96 VC:10 E [more]...
      
          int32_field_required TV=10 RL=0 DL=0
          ----------------------------------------------------------------------------
          page 0:  DLE:RLE RLE:RLE VLE:DELTA_BINARY_PACKED ST:[min: 0, max:  [more]... VC:10
      
          int32_field_optional TV=10 RL=0 DL=1
          ----------------------------------------------------------------------------
          page 0:  DLE:RLE RLE:RLE VLE:DELTA_BINARY_PACKED ST:[min: 1, max:  [more]... VC:10
      
          int32_field_repeated TV=10 RL=1 DL=1
          ----------------------------------------------------------------------------
          page 0:  DLE:RLE RLE:RLE VLE:DELTA_BINARY_PACKED ST:[min: 2, max:  [more]... VC:10
      
      INT32 int32_field_required
      --------------------------------------------------------------------------------
      *** row group 1 of 1, values 1 to 10 ***
      value 1:  R:0 D:0 V:0
      value 2:  R:0 D:0 V:3
      value 3:  R:0 D:0 V:6
      value 4:  R:0 D:0 V:9
      value 5:  R:0 D:0 V:12
      value 6:  R:0 D:0 V:15
      value 7:  R:0 D:0 V:18
      value 8:  R:0 D:0 V:21
      value 9:  R:0 D:0 V:24
      value 10: R:0 D:0 V:27
      
      INT32 int32_field_optional
      --------------------------------------------------------------------------------
      *** row group 1 of 1, values 1 to 10 ***
      value 1:  R:0 D:1 V:1
      value 2:  R:0 D:1 V:4
      value 3:  R:0 D:1 V:7
      value 4:  R:0 D:1 V:10
      value 5:  R:0 D:1 V:13
      value 6:  R:0 D:1 V:16
      value 7:  R:0 D:1 V:19
      value 8:  R:0 D:1 V:22
      value 9:  R:0 D:1 V:25
      value 10: R:0 D:1 V:28
      
      INT32 int32_field_repeated
      --------------------------------------------------------------------------------
      *** row group 1 of 1, values 1 to 10 ***
      value 1:  R:0 D:1 V:2
      value 2:  R:0 D:1 V:5
      value 3:  R:0 D:1 V:8
      value 4:  R:0 D:1 V:11
      value 5:  R:0 D:1 V:14
      value 6:  R:0 D:1 V:17
      value 7:  R:0 D:1 V:20
      value 8:  R:0 D:1 V:23
      value 9:  R:0 D:1 V:26
      value 10: R:0 D:1 V:29
      

      Here is the drillbit thread dump:

      [root@perfnode169 ~]# jstack -l 7355
      2016-07-06 17:25:56
      Full thread dump OpenJDK 64-Bit Server VM (24.79-b02 mixed mode):
      
      "Attach Listener" daemon prio=10 tid=0x00007fbae45a0800 nid=0x2a52 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-176" prio=10 tid=0x00000000016bd000 nid=0x2329 waiting on condition [0x00007fbab749a000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f7700410> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:513)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:48)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:569)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-174" prio=10 tid=0x0000000001b3c800 nid=0x2327 waiting on condition [0x00007fbabd7e5000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f7700410> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:513)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:48)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:569)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Scheduler-513671453" prio=10 tid=0x000000000150c800 nid=0x2326 waiting on condition [0x00007fbab76c1000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006e8034ae8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "288262b0-f8ad-46f3-2dda-ae4a64f67161:frag:0:0" daemon prio=10 tid=0x00000000013d9800 nid=0x32b runnable [0x00007fbabc5e1000]
         java.lang.Thread.State: RUNNABLE
      	at java.io.DataInputStream.readFully(DataInputStream.java:195)
      	at java.io.DataInputStream.readFully(DataInputStream.java:169)
      	at org.apache.parquet.bytes.BytesInput$StreamBytesInput.toByteArray(BytesInput.java:234)
      	at org.apache.parquet.bytes.BytesInput.toByteBuffer(BytesInput.java:185)
      	at org.apache.parquet.hadoop.ColumnChunkIncReadStore$ColumnChunkIncPageReader.readPage(ColumnChunkIncReadStore.java:195)
      	at org.apache.parquet.column.impl.ColumnReaderImpl.readPage(ColumnReaderImpl.java:537)
      	at org.apache.parquet.column.impl.ColumnReaderImpl.checkRead(ColumnReaderImpl.java:530)
      	at org.apache.parquet.column.impl.ColumnReaderImpl.consume(ColumnReaderImpl.java:642)
      	at org.apache.parquet.column.impl.ColumnReaderImpl.<init>(ColumnReaderImpl.java:358)
      	at org.apache.parquet.column.impl.ColumnReadStoreImpl.newMemColumnReader(ColumnReadStoreImpl.java:82)
      	at org.apache.parquet.column.impl.ColumnReadStoreImpl.getColumnReader(ColumnReadStoreImpl.java:77)
      	at org.apache.parquet.io.RecordReaderImplementation.<init>(RecordReaderImplementation.java:270)
      	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:140)
      	at org.apache.parquet.io.MessageColumnIO$1.visit(MessageColumnIO.java:106)
      	at org.apache.parquet.filter2.compat.FilterCompat$NoOpFilter.accept(FilterCompat.java:154)
      	at org.apache.parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:106)
      	at org.apache.parquet.io.MessageColumnIO.getRecordReader(MessageColumnIO.java:82)
      	at org.apache.drill.exec.store.parquet2.DrillParquetReader.setup(DrillParquetReader.java:268)
      	at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:101)
      	at org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:140)
      	at org.apache.drill.exec.store.parquet.ParquetScanBatchCreator.getBatch(ParquetScanBatchCreator.java:53)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:148)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:171)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:128)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:171)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:101)
      	at org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:79)
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:231)
      	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- <0x00000006fb063ef8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
      
      "BitServer-4" daemon prio=10 tid=0x00007fbae45a1800 nid=0x1fa5 runnable [0x00007fbab7193000]
         java.lang.Thread.State: RUNNABLE
      	at io.netty.channel.epoll.Native.epollWait0(Native Method)
      	at io.netty.channel.epoll.Native.epollWait(Native.java:148)
      	at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "BitServer-3" daemon prio=10 tid=0x0000000001b22000 nid=0x1fa4 runnable [0x00007fbab7294000]
         java.lang.Thread.State: RUNNABLE
      	at io.netty.channel.epoll.Native.epollWait0(Native Method)
      	at io.netty.channel.epoll.Native.epollWait(Native.java:148)
      	at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-PathChildrenCache-2" daemon prio=10 tid=0x00007fbadc15f800 nid=0x1f62 waiting on condition [0x00007fbabc6e3000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006e9ba1680> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "threadDeathWatcher-2-1" daemon prio=10 tid=0x00007fbadc061800 nid=0x1f22 waiting on condition [0x00007fbabd8e6000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:137)
      	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-73-acceptor-1-ServerConnector@76b504b4{HTTP/1.1}{0.0.0.0:8047}" prio=10 tid=0x00007fbadc045800 nid=0x1d8a runnable [0x00007fbabd9e7000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
      	at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
      	- locked <0x00000006e8034228> (a java.lang.Object)
      	at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:336)
      	at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:467)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-72-acceptor-0-ServerConnector@76b504b4{HTTP/1.1}{0.0.0.0:8047}" prio=10 tid=0x000000000189f000 nid=0x1d89 waiting for monitor entry [0x00007fbabdae8000]
         java.lang.Thread.State: BLOCKED (on object monitor)
      	at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:225)
      	- waiting to lock <0x00000006e8034228> (a java.lang.Object)
      	at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:336)
      	at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:467)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-71-selector-ServerConnectorManager@40d64c4d/23" prio=10 tid=0x00007fbb2ef5a800 nid=0x1d88 runnable [0x00007fbabdbe9000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e82300c0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e82300b0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822ff98> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-70-selector-ServerConnectorManager@40d64c4d/22" prio=10 tid=0x00007fbae4032800 nid=0x1d87 runnable [0x00007fbabdcea000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822f2d0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822f2c0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822f1a8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-69-selector-ServerConnectorManager@40d64c4d/21" prio=10 tid=0x00007fbadc043800 nid=0x1d86 runnable [0x00007fbabddeb000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822e4e0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822e4d0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822e3b8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-68-selector-ServerConnectorManager@40d64c4d/20" prio=10 tid=0x0000000001dc6000 nid=0x1d85 runnable [0x00007fbabdeec000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822d6f0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822d6e0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822d5c8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-67-selector-ServerConnectorManager@40d64c4d/19" prio=10 tid=0x00007fbb2ef58800 nid=0x1d84 runnable [0x00007fbabdfed000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822c900> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822c8f0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822c7d8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-66-selector-ServerConnectorManager@40d64c4d/18" prio=10 tid=0x00007fbae4030800 nid=0x1d83 runnable [0x00007fbabe0ee000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822bb10> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822bb00> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822b9e8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-65-selector-ServerConnectorManager@40d64c4d/17" prio=10 tid=0x00007fbadc041800 nid=0x1d82 runnable [0x00007fbabe1ef000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e822acd0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e822acc0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e822aba8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-64-selector-ServerConnectorManager@40d64c4d/16" prio=10 tid=0x00000000016d8000 nid=0x1d81 runnable [0x00007fbabe2f0000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e81ffe18> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e81ffe08> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e81ffcf0> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-63-selector-ServerConnectorManager@40d64c4d/15" prio=10 tid=0x00007fbb2e5e4000 nid=0x1d80 runnable [0x00007fbabe3f1000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e81ff838> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e81ff828> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e81ff710> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "DestroyJavaVM" prio=10 tid=0x00007fbb2c012800 nid=0x1ce8 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-61-selector-ServerConnectorManager@40d64c4d/14" prio=10 tid=0x00007fbae402e800 nid=0x1d7f runnable [0x00007fbabe4f2000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80cc5a0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80cc590> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80cc478> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-59-selector-ServerConnectorManager@40d64c4d/13" prio=10 tid=0x00007fbadc03f800 nid=0x1d7e runnable [0x00007fbabe5f3000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80cbfc0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80cbfb0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80cbe98> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-58-selector-ServerConnectorManager@40d64c4d/12" prio=10 tid=0x00000000016d6800 nid=0x1d7d runnable [0x00007fbabe6f4000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80cb1d0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80cb1c0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80cb0a8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-57-selector-ServerConnectorManager@40d64c4d/11" prio=10 tid=0x00007fbb2e5e3000 nid=0x1d7c runnable [0x00007fbabe7f5000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80ca3e0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80ca3d0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80ca2b8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-56-selector-ServerConnectorManager@40d64c4d/10" prio=10 tid=0x00007fbae402c800 nid=0x1d7b runnable [0x00007fbabe8f6000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c95f0> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c95e0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c94c8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-55-selector-ServerConnectorManager@40d64c4d/9" prio=10 tid=0x00007fbadc033000 nid=0x1d7a runnable [0x00007fbabe9f7000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c8800> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c87f0> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c86d8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-54-selector-ServerConnectorManager@40d64c4d/8" prio=10 tid=0x00007fbb2dd08000 nid=0x1d79 runnable [0x00007fbabeaf8000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c7a10> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c7a00> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c78e8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Scheduler-871579151" prio=10 tid=0x00007fbb2df6b800 nid=0x1d78 waiting on condition [0x00007fbabebf9000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f7700278> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-52-selector-ServerConnectorManager@40d64c4d/7" prio=10 tid=0x00007fbb2ce12800 nid=0x1d77 runnable [0x00007fbabecfa000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c6c20> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c6c10> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c6af8> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-51-selector-ServerConnectorManager@40d64c4d/6" prio=10 tid=0x00007fbb2ce10800 nid=0x1d76 runnable [0x00007fbabedfb000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c5e30> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c5e20> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c5d08> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-50-selector-ServerConnectorManager@40d64c4d/4" prio=10 tid=0x00007fbb2d8df800 nid=0x1d75 runnable [0x00007fbabf08b000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c4250> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c4240> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c4128> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-49-selector-ServerConnectorManager@40d64c4d/5" prio=10 tid=0x00007fbb2d8dd800 nid=0x1d74 runnable [0x00007fbabf18c000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c5040> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c5030> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c4f18> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-48-selector-ServerConnectorManager@40d64c4d/3" prio=10 tid=0x00007fbb2d8db800 nid=0x1d73 runnable [0x00007fbabf28d000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c3460> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c3450> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c3338> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-47-selector-ServerConnectorManager@40d64c4d/2" prio=10 tid=0x00007fbb2d8da000 nid=0x1d72 runnable [0x00007fbabf38e000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80c2670> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80c2660> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80c2548> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-46-selector-ServerConnectorManager@40d64c4d/1" prio=10 tid=0x00007fbb2e126800 nid=0x1d71 runnable [0x00007fbabf48f000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e80b1638> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e80b1628> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e80b1510> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "qtp239614979-45-selector-ServerConnectorManager@40d64c4d/0" prio=10 tid=0x00007fbb2e123800 nid=0x1d70 runnable [0x00007fbabf7a4000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006e808be08> (a sun.nio.ch.Util$2)
      	- locked <0x00000006e808bdf8> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006e808bce0> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.select(SelectorManager.java:532)
      	at org.eclipse.jetty.io.SelectorManager$ManagedSelector.run(SelectorManager.java:485)
      	at org.eclipse.jetty.util.thread.NonBlockingThread.run(NonBlockingThread.java:52)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-PathChildrenCache-1" daemon prio=10 tid=0x00007fbb2eac5800 nid=0x1d6f waiting on condition [0x00007fbabeefc000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f7600068> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "WorkManager.StatusThread" daemon prio=10 tid=0x00007fbb2eb7f000 nid=0x1d6a waiting on condition [0x00007fbabf590000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.drill.exec.work.WorkManager$StatusThread.run(WorkManager.java:318)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-PathChildrenCache-0" daemon prio=10 tid=0x00007fbb2ce85000 nid=0x1d69 waiting on condition [0x00007fbabf691000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f83046c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "BitServer-2" daemon prio=10 tid=0x00007fbb2e41e000 nid=0x1d68 runnable [0x00007fbabf8a5000]
         java.lang.Thread.State: RUNNABLE
      	at io.netty.channel.epoll.Native.epollWait0(Native Method)
      	at io.netty.channel.epoll.Native.epollWait(Native.java:148)
      	at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "BitServer-1" daemon prio=10 tid=0x00007fbb2e051800 nid=0x1d67 runnable [0x00007fbabf9a6000]
         java.lang.Thread.State: RUNNABLE
      	at io.netty.channel.epoll.Native.epollWait0(Native Method)
      	at io.netty.channel.epoll.Native.epollWait(Native.java:148)
      	at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "UserServer-1" daemon prio=10 tid=0x00007fbb2d949800 nid=0x1d66 runnable [0x00007fbabfaa7000]
         java.lang.Thread.State: RUNNABLE
      	at io.netty.channel.epoll.Native.epollWait0(Native Method)
      	at io.netty.channel.epoll.Native.epollWait(Native.java:148)
      	at io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:180)
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:205)
      	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-ServiceCache-0" daemon prio=10 tid=0x00007fbb2e05f000 nid=0x1d64 waiting on condition [0x00007fbabfda9000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f8304828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-Framework-0" daemon prio=10 tid=0x00007fbb2e328800 nid=0x1d22 waiting on condition [0x00007fbac1020000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f8304990> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.DelayQueue.take(DelayQueue.java:209)
      	at java.util.concurrent.DelayQueue.take(DelayQueue.java:68)
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:780)
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:62)
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:257)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- <0x00000006f8304ad0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
      
      "main-EventThread" daemon prio=10 tid=0x00007fbb2e31e000 nid=0x1d21 waiting on condition [0x00007fbac1121000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f76006c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
      	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
      
         Locked ownable synchronizers:
      	- None
      
      "main-SendThread(10.10.30.168:5181)" daemon prio=10 tid=0x00007fbb2e318800 nid=0x1d20 runnable [0x00007fbac1222000]
         java.lang.Thread.State: RUNNABLE
      	at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
      	at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
      	at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
      	at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
      	- locked <0x00000006f7600868> (a sun.nio.ch.Util$2)
      	- locked <0x00000006f7600878> (a java.util.Collections$UnmodifiableSet)
      	- locked <0x00000006f7600820> (a sun.nio.ch.EPollSelectorImpl)
      	at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
      	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
      	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1070)
      
         Locked ownable synchronizers:
      	- None
      
      "Curator-ConnectionStateManager-0" daemon prio=10 tid=0x00007fbb2e33e000 nid=0x1d1f waiting on condition [0x00007fbac1323000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000006f8304b68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
      	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374)
      	at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:245)
      	at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:43)
      	at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:111)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
         Locked ownable synchronizers:
      	- <0x00000006f8304c80> (a java.util.concurrent.ThreadPoolExecutor$Worker)
      
      "Service Thread" daemon prio=10 tid=0x00007fbb2cdcd800 nid=0x1d1b runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C2 CompilerThread1" daemon prio=10 tid=0x00007fbb2cdca800 nid=0x1d1a waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "C2 CompilerThread0" daemon prio=10 tid=0x00007fbb2cdc8000 nid=0x1d19 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Signal Dispatcher" daemon prio=10 tid=0x00007fbb2cdc6000 nid=0x1d18 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x00007fbb2cdc4000 nid=0x1d17 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
         Locked ownable synchronizers:
      	- None
      
      "Finalizer" daemon prio=10 tid=0x00007fbb2cd96000 nid=0x1d16 in Object.wait() [0x00007fbae10cf000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000006e7e01908> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
      	- locked <0x00000006e7e01908> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
      
         Locked ownable synchronizers:
      	- None
      
      "Reference Handler" daemon prio=10 tid=0x00007fbb2cd8e800 nid=0x1d15 in Object.wait() [0x00007fbae11d0000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000006e7e018f8> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:503)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
      	- locked <0x00000006e7e018f8> (a java.lang.ref.Reference$Lock)
      
         Locked ownable synchronizers:
      	- None
      
      "VM Thread" prio=10 tid=0x00007fbb2cd8a000 nid=0x1d14 runnable
      
      "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c027000 nid=0x1ce9 runnable
      
      "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c029000 nid=0x1cea runnable
      
      "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c02a800 nid=0x1ceb runnable
      
      "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c02c800 nid=0x1cec runnable
      
      "Gang worker#4 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c02e800 nid=0x1ced runnable
      
      "Gang worker#5 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c030800 nid=0x1cee runnable
      
      "Gang worker#6 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c032000 nid=0x1cef runnable
      
      "Gang worker#7 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c034000 nid=0x1cf0 runnable
      
      "Gang worker#8 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c036000 nid=0x1cf1 runnable
      
      "Gang worker#9 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c037800 nid=0x1cf2 runnable
      
      "Gang worker#10 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c039800 nid=0x1cf3 runnable
      
      "Gang worker#11 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c03b800 nid=0x1cf4 runnable
      
      "Gang worker#12 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c03d000 nid=0x1cf5 runnable
      
      "Gang worker#13 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c03f000 nid=0x1cf6 runnable
      
      "Gang worker#14 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c041000 nid=0x1cf7 runnable
      
      "Gang worker#15 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c043000 nid=0x1cf8 runnable
      
      "Gang worker#16 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c044800 nid=0x1cf9 runnable
      
      "Gang worker#17 (Parallel GC Threads)" prio=10 tid=0x00007fbb2c046800 nid=0x1cfa runnable
      
      "G1 Main Concurrent Mark GC Thread" prio=10 tid=0x00007fbb2c076800 nid=0x1d0e runnable
      
      "Gang worker#0 (G1 Parallel Marking Threads)" prio=10 tid=0x00007fbb2c0c3000 nid=0x1d0f runnable
      
      "Gang worker#1 (G1 Parallel Marking Threads)" prio=10 tid=0x00007fbb2c0c4800 nid=0x1d10 runnable
      
      "Gang worker#2 (G1 Parallel Marking Threads)" prio=10 tid=0x00007fbb2c0c6800 nid=0x1d11 runnable
      
      "Gang worker#3 (G1 Parallel Marking Threads)" prio=10 tid=0x00007fbb2c0c8800 nid=0x1d12 runnable
      
      "Gang worker#4 (G1 Parallel Marking Threads)" prio=10 tid=0x00007fbb2c0ca800 nid=0x1d13 runnable
      
      "G1 Concurrent Refinement Thread#0" prio=10 tid=0x00007fbb2c06c000 nid=0x1d0d runnable
      
      "G1 Concurrent Refinement Thread#1" prio=10 tid=0x00007fbb2c06a800 nid=0x1d0c runnable
      
      "G1 Concurrent Refinement Thread#2" prio=10 tid=0x00007fbb2c068800 nid=0x1d0b runnable
      
      "G1 Concurrent Refinement Thread#3" prio=10 tid=0x00007fbb2c066800 nid=0x1d0a runnable
      
      "G1 Concurrent Refinement Thread#4" prio=10 tid=0x00007fbb2c064800 nid=0x1d09 runnable
      
      "G1 Concurrent Refinement Thread#5" prio=10 tid=0x00007fbb2c063000 nid=0x1d08 runnable
      
      "G1 Concurrent Refinement Thread#6" prio=10 tid=0x00007fbb2c061000 nid=0x1d07 runnable
      
      "G1 Concurrent Refinement Thread#7" prio=10 tid=0x00007fbb2c05f000 nid=0x1d06 runnable
      
      "G1 Concurrent Refinement Thread#8" prio=10 tid=0x00007fbb2c05d000 nid=0x1d05 runnable
      
      "G1 Concurrent Refinement Thread#9" prio=10 tid=0x00007fbb2c05b800 nid=0x1d04 runnable
      
      "G1 Concurrent Refinement Thread#10" prio=10 tid=0x00007fbb2c059800 nid=0x1d03 runnable
      
      "G1 Concurrent Refinement Thread#11" prio=10 tid=0x00007fbb2c057800 nid=0x1d02 runnable
      
      "G1 Concurrent Refinement Thread#12" prio=10 tid=0x00007fbb2c055800 nid=0x1d01 runnable
      
      "G1 Concurrent Refinement Thread#13" prio=10 tid=0x00007fbb2c054000 nid=0x1d00 runnable
      
      "G1 Concurrent Refinement Thread#14" prio=10 tid=0x00007fbb2c052000 nid=0x1cff runnable
      
      "G1 Concurrent Refinement Thread#15" prio=10 tid=0x00007fbb2c050000 nid=0x1cfe runnable
      
      "G1 Concurrent Refinement Thread#16" prio=10 tid=0x00007fbb2c04e000 nid=0x1cfd runnable
      
      "G1 Concurrent Refinement Thread#17" prio=10 tid=0x00007fbb2c04c800 nid=0x1cfc runnable
      
      "G1 Concurrent Refinement Thread#18" prio=10 tid=0x00007fbb2c04a800 nid=0x1cfb runnable
      
      
      "VM Periodic Task Thread" prio=10 tid=0x00007fbb2ce0a800 nid=0x1d1c waiting on condition
      
      JNI global references: 401
      

      Attachments

        Activity

          People

            ppenumarthy Padma Penumarthy
            cchang@maprtech.com Chun Chang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: