[2018-03-06 14:00:49,585] INFO Rolled new log segment for 'app_running-30' in 1 ms. (kafka.log.Log)
[2018-03-06 14:01:50,564] INFO Rolled new log segment for 'app_running-34' in 1 ms. (kafka.log.Log)
[2018-03-06 14:02:01,565] INFO Rolled new log segment for 'user_location-8' in 2 ms. (kafka.log.Log)
[2018-03-06 14:02:06,702] INFO [ReplicaFetcherManager on broker 58] Removed fetcher for partitions __consumer_offsets-63 (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:02:06,702] INFO Partition [__consumer_offsets,63] on broker 58: __consumer_offsets-63 starts at Leader Epoch 491 from offset 794112852. Previous Leader Epoch was: 490 (kafka.cluster.Partition)
[2018-03-06 14:02:06,702] INFO [Group Metadata Manager on Broker 58]: Scheduling loading of offsets and group metadata from __consumer_offsets-63 (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:02:06,928] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:06,928] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:06,928] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,682] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,682] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,682] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,720] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,720] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,720] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,727] INFO [ReplicaFetcherManager on broker 58] Removed fetcher for partitions ctm_ehouse_report-38 (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:02:07,728] INFO Partition [ctm_ehouse_report,38] on broker 58: ctm_ehouse_report-38 starts at Leader Epoch 154 from offset 2. Previous Leader Epoch was: 153 (kafka.cluster.Partition)
[2018-03-06 14:02:07,759] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,759] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,759] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,797] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,797] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,797] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,838] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,839] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,839] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,845] INFO [ReplicaFetcherManager on broker 58] Removed fetcher for partitions __consumer_offsets-2 (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:02:07,845] INFO [ReplicaFetcherManager on broker 58] Added fetcher for partitions List([__consumer_offsets-2, initOffset 2789766206 to broker BrokerEndPoint(57,172.16.99.108,9092)] ) (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:02:07,846] INFO [ReplicaFetcherThread-0-1]: Shutting down (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:02:07,847] INFO Partition [ctm_ehouse_report,38] on broker 58: Expanding ISR from 2,58 to 2,58,1 (kafka.cluster.Partition)
[2018-03-06 14:02:07,886] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,886] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,886] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,929] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,930] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,930] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:07,971] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
......
......
[2018-03-06 14:02:41,288] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,326] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,326] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,326] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,365] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,365] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,365] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,402] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,402] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,402] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,435] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.84.127 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,435] INFO [GroupCoordinator 58]: Loading group metadata for flume.mpns with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,435] INFO [GroupCoordinator 58]: Loading group metadata for command-172.16.83.245 with generation 0 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,435] INFO [Group Metadata Manager on Broker 58]: Finished loading offsets and group metadata from __consumer_offsets-63 in 34733 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:02:41,435] INFO [Group Metadata Manager on Broker 58]: Finished unloading __consumer_offsets-2. Removed 0 cached offsets and 0 cached groups. (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:02:41,463] INFO [GroupCoordinator 58]: Preparing to rebalance group command-172.16.84.127 with old generation 0 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:41,476] INFO [GroupCoordinator 58]: Preparing to rebalance group command-172.16.83.245 with old generation 0 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:43,745] INFO [GroupCoordinator 58]: Preparing to rebalance group flume.mpns with old generation 0 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:44,466] INFO [GroupCoordinator 58]: Stabilized group command-172.16.84.127 generation 1 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:44,467] INFO [GroupCoordinator 58]: Assignment received from leader for group command-172.16.84.127 for generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:44,477] INFO [GroupCoordinator 58]: Stabilized group command-172.16.83.245 generation 1 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:44,478] INFO [GroupCoordinator 58]: Assignment received from leader for group command-172.16.83.245 for generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:46,747] INFO [GroupCoordinator 58]: Stabilized group flume.mpns generation 1 (__consumer_offsets-63) (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:02:46,750] INFO [GroupCoordinator 58]: Assignment received from leader for group flume.mpns for generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:03:27,678] INFO Scheduling log segment 1049176658 for log app-list-14 for deletion. (kafka.log.Log)
[2018-03-06 14:04:27,679] INFO Deleting segment 1049176658 from log app-list-14. (kafka.log.Log)
[2018-03-06 14:04:27,679] INFO Deleting index /opt/push/kafka/data/app-list-14/00000000001049176658.index.deleted (kafka.log.OffsetIndex)
[2018-03-06 14:04:27,679] INFO Deleting index /opt/push/kafka/data/app-list-14/00000000001049176658.timeindex.deleted (kafka.log.TimeIndex)
[2018-03-06 14:04:34,504] INFO Partition [ctm_ehouse_report,38] on broker 58: Expanding ISR from 2,58 to 2,58,1 (kafka.cluster.Partition)
[2018-03-06 14:04:34,783] INFO Partition [__consumer_offsets,63] on broker 58: Expanding ISR from 58,59 to 58,59,1 (kafka.cluster.Partition)
[2018-03-06 14:04:35,246] INFO Partition [crash,63] on broker 58: Expanding ISR from 58 to 58,1 (kafka.cluster.Partition)
[2018-03-06 14:05:59,366] INFO Rolled new log segment for 'user_location-40' in 2 ms. (kafka.log.Log)
[2018-03-06 14:07:00,745] INFO [ReplicaFetcherManager on broker 58] Removed fetcher for partitions __consumer_offsets-63 (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:07:00,747] INFO [ReplicaFetcherThread-0-1]: Starting (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:07:00,747] INFO [ReplicaFetcherManager on broker 58] Added fetcher for partitions List([__consumer_offsets-63, initOffset 794129204 to broker BrokerEndPoint(1,172.16.85.107,9092)] ) (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:07:00,747] INFO [Group Metadata Manager on Broker 58]: Scheduling unloading of offsets and group metadata from __consumer_offsets-63 (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:07:00,747] INFO [GroupCoordinator 58]: Unloading group metadata for command-172.16.84.127 with generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:07:00,748] INFO [GroupCoordinator 58]: Unloading group metadata for command-172.16.83.245 with generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:07:00,748] INFO [GroupCoordinator 58]: Unloading group metadata for flume.mpns with generation 1 (kafka.coordinator.group.GroupCoordinator)
[2018-03-06 14:07:00,748] INFO [Group Metadata Manager on Broker 58]: Finished unloading __consumer_offsets-63. Removed 72 cached offsets and 3 cached groups. (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:07:00,748] INFO [ReplicaFetcherThread-0-1]: Retrying leaderEpoch request for partition __consumer_offsets-63 as the leader reported an error: NOT_LEADER_FOR_PARTITION (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:07:01,701] INFO [ReplicaFetcherManager on broker 58] Removed fetcher for partitions __consumer_offsets-2 (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:07:01,701] INFO [ReplicaFetcherManager on broker 58] Added fetcher for partitions List([__consumer_offsets-2, initOffset 2789766206 to broker BrokerEndPoint(1,172.16.85.107,9092)] ) (kafka.server.ReplicaFetcherManager)
[2018-03-06 14:07:01,701] INFO [Group Metadata Manager on Broker 58]: Scheduling unloading of offsets and group metadata from __consumer_offsets-2 (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:07:01,701] INFO [Group Metadata Manager on Broker 58]: Finished unloading __consumer_offsets-2. Removed 0 cached offsets and 0 cached groups. (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:07:01,734] INFO [ReplicaFetcherThread-0-1]: Retrying leaderEpoch request for partition __consumer_offsets-2 as the leader reported an error: NOT_LEADER_FOR_PARTITION (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:07:02,766] WARN [ReplicaFetcherThread-0-1]: Based on follower's leader epoch, leader replied with an unknown offset in __consumer_offsets-2. High watermark 2789766206 will be used for truncation. (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:07:02,766] INFO [ReplicaFetcherThread-0-1]: Based on follower's leader epoch, leader replied with an offset 794129204 >= the follower's log end offset 794129204 in __consumer_offsets-63. No truncation needed. (kafka.server.ReplicaFetcherThread)
[2018-03-06 14:07:02,766] INFO Truncating __consumer_offsets-2 to 2789766206 has no effect as the largest offset in the log is 2789766205. (kafka.log.Log)
[2018-03-06 14:07:02,767] INFO Truncating __consumer_offsets-63 to 794129204 has no effect as the largest offset in the log is 794129203. (kafka.log.Log)
[2018-03-06 14:07:04,844] INFO Rolled new log segment for 'app_running-31' in 2 ms. (kafka.log.Log)
[2018-03-06 14:07:58,251] INFO [Group Metadata Manager on Broker 58]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2018-03-06 14:08:01,237] INFO Rolled new log segment for 'app_running-30' in 1 ms. (kafka.log.Log)
[2018-03-06 14:08:27,677] INFO Scheduling log segment 26810869663 for log user_location-4 for deletion. (kafka.log.Log)
[2018-03-06 14:08:27,679] INFO Scheduling log segment 3673424306 for log lbs_info_simplified-37 for deletion. (kafka.log.Log)
[2018-03-06 14:08:27,681] INFO Scheduling log segment 2683844008 for log app_running-34 for deletion. (kafka.log.Log)
[2018-03-06 14:08:27,684] INFO Scheduling log segment 2681259779 for log app_running-31 for deletion. (kafka.log.Log)
[2018-03-06 14:08:27,687] INFO Scheduling log segment 13521387291 for log lbs_info_all-44 for deletion. (kafka.log.Log)
[2018-03-06 14:08:27,689] INFO Scheduling log segment 2683085405 for log app_running-30 for deletion. (kafka.log.Log)
[2018-03-06 14:09:03,480] INFO Rolled new log segment for 'app_running-34' in 1 ms. (kafka.log.Log)
[2018-03-06 14:09:27,677] INFO Deleting segment 26810869663 from log user_location-4. (kafka.log.Log)
[2018-03-06 14:09:27,677] INFO Deleting index /opt/push/kafka/data/user_location-4/00000000026810869663.index.deleted (kafka.log.OffsetIndex)