Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8123

Flaky Test RequestQuotaTest#testResponseThrottleTimeWhenBothProduceAndRequestQuotasViolated

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Duplicate
    • 2.3.0
    • None
    • core, unit tests

    Description

      https://builds.apache.org/blue/organizations/jenkins/kafka-trunk-jdk8/detail/kafka-trunk-jdk8/3474/tests

      java.util.concurrent.ExecutionException: java.lang.AssertionError: Throttle time metrics for produce quota not updated: Client small-quota-producer-client apiKey PRODUCE requests 1 requestTime 0.015790873650539786 throttleTime 1000.0
      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      at java.util.concurrent.FutureTask.get(FutureTask.java:206)
      at kafka.server.RequestQuotaTest$$anonfun$waitAndCheckResults$1.apply(RequestQuotaTest.scala:423)
      at kafka.server.RequestQuotaTest$$anonfun$waitAndCheckResults$1.apply(RequestQuotaTest.scala:421)
      at scala.collection.immutable.List.foreach(List.scala:392)
      at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
      at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:45)
      at kafka.server.RequestQuotaTest.waitAndCheckResults(RequestQuotaTest.scala:421)
      at kafka.server.RequestQuotaTest.testResponseThrottleTimeWhenBothProduceAndRequestQuotasViolated(RequestQuotaTest.scala:130)

      STDOUT

      [2019-03-18 21:42:16,637] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CONTROLLED_SHUTDOWN, correlationId=1, api=CONTROLLED_SHUTDOWN, body={broker_id=0,broker_epoch=9223372036854775807} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47612-1, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,655] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-STOP_REPLICA, correlationId=1, api=STOP_REPLICA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,delete_partitions=true,partitions=[

      Unknown macro: {topic=topic-1,partition_ids=[0]}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=127.0.0.1:42118-127.0.0.1:47614-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,657] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-LEADER_AND_ISR, correlationId=1, api=LEADER_AND_ISR, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      Unknown macro: {partition=0,controller_epoch=2147483647,leader=0,leader_epoch=2147483647,isr=[0],zk_version=2,replicas=[0],is_new=true}

      ]}],live_leaders=[\{id=0,host=localhost,port=0}]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47616-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,668] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-UPDATE_METADATA, correlationId=1, api=UPDATE_METADATA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      Unknown macro: {partition=0,controller_epoch=2147483647,leader=0,leader_epoch=2147483647,isr=[0],zk_version=2,replicas=[0],offline_replicas=[]}

      ]}],live_brokers=[{id=0,end_points=[

      Unknown macro: {port=0,host=localhost,listener_name=PLAINTEXT,security_protocol_type=0}

      ],rack=null}]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47618-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,725] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-STOP_REPLICA, correlationId=2, api=STOP_REPLICA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,delete_partitions=true,partitions=[

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=127.0.0.1:42118-127.0.0.1:47614-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,726] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-LEADER_AND_ISR, correlationId=2, api=LEADER_AND_ISR, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      Unknown macro: {partition=0,controller_epoch=2147483647,leader=0,leader_epoch=2147483647,isr=[0],zk_version=2,replicas=[0],is_new=true}

      ]}],live_leaders=[\{id=0,host=localhost,port=0}]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47616-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,948] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DELETE_ACLS, correlationId=1, api=DELETE_ACLS, body={filters=[

      Unknown macro: {resource_type=2,resource_name=null,resource_pattern_type_filter=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47666-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,943] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-UPDATE_METADATA, correlationId=2, api=UPDATE_METADATA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      Unknown macro: {partition=0,controller_epoch=2147483647,leader=0,leader_epoch=2147483647,isr=[0],zk_version=2,replicas=[0],offline_replicas=[]}

      ]}],live_brokers=[{id=0,end_points=[

      Unknown macro: {port=0,host=localhost,listener_name=PLAINTEXT,security_protocol_type=0}

      ],rack=null}]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47618-2, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,941] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CONTROLLED_SHUTDOWN, correlationId=2, api=CONTROLLED_SHUTDOWN, body={broker_id=0,broker_epoch=9223372036854775807} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47612-1, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,961] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-WRITE_TXN_MARKERS, correlationId=1, api=WRITE_TXN_MARKERS, body={transaction_markers=[]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47658-9, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,962] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CREATE_ACLS, correlationId=1, api=CREATE_ACLS, body={creations=[

      Unknown macro: {resource_type=2,resource_name=mytopic,resource_pattten_type=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47664-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,963] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DESCRIBE_ACLS, correlationId=1, api=DESCRIBE_ACLS, body={resource_type=1,resource_name=null,resource_pattern_type_filter=1,principal=null,host=null,operation=1,permission_type=1} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=127.0.0.1:42118-127.0.0.1:47662-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,973] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DESCRIBE_ACLS, correlationId=2, api=DESCRIBE_ACLS, body={resource_type=1,resource_name=null,resource_pattern_type_filter=1,principal=null,host=null,operation=1,permission_type=1} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=127.0.0.1:42118-127.0.0.1:47662-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,973] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-WRITE_TXN_MARKERS, correlationId=2, api=WRITE_TXN_MARKERS, body={transaction_markers=[]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47658-9, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:16,974] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CREATE_ACLS, correlationId=2, api=CREATE_ACLS, body={creations=[

      Unknown macro: {resource_type=2,resource_name=mytopic,resource_pattten_type=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47664-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,005] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DELETE_ACLS, correlationId=2, api=DELETE_ACLS, body={filters=[

      Unknown macro: {resource_type=2,resource_name=null,resource_pattern_type_filter=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47666-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,019] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DESCRIBE_ACLS, correlationId=3, api=DESCRIBE_ACLS, body={resource_type=1,resource_name=null,resource_pattern_type_filter=1,principal=null,host=null,operation=1,permission_type=1} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=0, connectionId=127.0.0.1:42118-127.0.0.1:47662-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,021] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-WRITE_TXN_MARKERS, correlationId=3, api=WRITE_TXN_MARKERS, body={transaction_markers=[]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47658-9, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,021] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CREATE_ACLS, correlationId=3, api=CREATE_ACLS, body={creations=[

      Unknown macro: {resource_type=2,resource_name=mytopic,resource_pattten_type=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47664-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,048] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-DELETE_ACLS, correlationId=3, api=DELETE_ACLS, body={filters=[

      Unknown macro: {resource_type=2,resource_name=null,resource_pattern_type_filter=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=2, connectionId=127.0.0.1:42118-127.0.0.1:47666-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,068] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-WRITE_TXN_MARKERS, correlationId=4, api=WRITE_TXN_MARKERS, body={transaction_markers=[]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47658-9, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:17,068] ERROR [KafkaApi-0] Error when handling request: clientId=unauthorized-CREATE_ACLS, correlationId=4, api=CREATE_ACLS, body={creations=[

      Unknown macro: {resource_type=2,resource_name=mytopic,resource_pattten_type=3,principal=User}

      ]} (kafka.server.KafkaApis:76)
      org.apache.kafka.common.errors.ClusterAuthorizationException: Request Request(processor=1, connectionId=127.0.0.1:42118-127.0.0.1:47664-10, session=Session(User:Unauthorized,/127.0.0.1), listenerName=ListenerName(PLAINTEXT), securityProtocol=PLAINTEXT, buffer=null) is not authorized.
      [2019-03-18 21:42:24,338] WARN Unable to read additional data from client sessionid 0x10241c03e640000, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn:376)
      [2019-03-18 21:42:26,379] ERROR [KafkaApi-0] Error when handling request: clientId=LEADER_AND_ISR, correlationId=1, api=LEADER_AND_ISR, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      ]}],live_leaders=[\{id=0,host=localhost,port=0}]} (kafka.server.KafkaApis:76)
      java.lang.IllegalStateException: Epoch 9223372036854775807 larger than current broker epoch 25
      at kafka.server.KafkaApis.isBrokerEpochStale(KafkaApis.scala:2470)
      at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:193)
      at kafka.server.KafkaApis.handle(KafkaApis.scala:117)
      at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:69)
      at java.lang.Thread.run(Thread.java:748)
      [2019-03-18 21:42:26,381] ERROR [KafkaApi-0] Error when handling request: clientId=UPDATE_METADATA, correlationId=1, api=UPDATE_METADATA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,topic_states=[{topic=topic-1,partition_states=[

      Unknown macro: {partition=0,controller_epoch=2147483647,leader=0,leader_epoch=2147483647,isr=[0],zk_version=2,replicas=[0],offline_replicas=[]}

      ]}],live_brokers=[{id=0,end_points=[

      Unknown macro: {port=0,host=localhost,listener_name=PLAINTEXT,security_protocol_type=0}

      ],rack=null}]} (kafka.server.KafkaApis:76)
      java.lang.IllegalStateException: Epoch 9223372036854775807 larger than current broker epoch 25
      at kafka.server.KafkaApis.isBrokerEpochStale(KafkaApis.scala:2470)
      at kafka.server.KafkaApis.handleUpdateMetadataRequest(KafkaApis.scala:241)
      at kafka.server.KafkaApis.handle(KafkaApis.scala:119)
      at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:69)
      at java.lang.Thread.run(Thread.java:748)
      [2019-03-18 21:42:26,380] ERROR [KafkaApi-0] Error when handling request: clientId=STOP_REPLICA, correlationId=1, api=STOP_REPLICA, body={controller_id=0,controller_epoch=2147483647,broker_epoch=9223372036854775807,delete_partitions=true,partitions=[

      Unknown macro: {topic=topic-1,partition_ids=[0]}

      ]} (kafka.server.KafkaApis:76)
      java.lang.IllegalStateException: Epoch 9223372036854775807 larger than current broker epoch 25
      at kafka.server.KafkaApis.isBrokerEpochStale(KafkaApis.scala:2470)
      at kafka.server.KafkaApis.handleStopReplicaRequest(KafkaApis.scala:211)
      at kafka.server.KafkaApis.handle(KafkaApis.scala:118)
      at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:69)
      at java.lang.Thread.run(Thread.java:748)
      [2019-03-18 21:42:26,971] WARN Unable to read additional data from client sessionid 0x10241c04b2c0001, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn:376)
      [2019-03-18 21:42:34,709] ERROR Test failed for api-key PRODUCE with exception java.util.concurrent.ExecutionException: java.lang.AssertionError: Throttle time metrics for produce quota not updated: Client small-quota-producer-client apiKey PRODUCE requests 1 requestTime 0.015790873650539786 throttleTime 1000.0 (kafka.server.RequestQuotaTest:74)
      [2019-03-18 21:42:35,746] WARN Unable to read additional data from client sessionid 0x10241c06b4f0001, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn:376)

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-7989 Flaky Test RequestQuotaTest#testResponseThrottleTimeWhenBothFetchAndRequestQuotasViolated

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-7989 Flaky Test RequestQuotaTest#testResponseThrottleTimeWhenBothFetchAndRequestQuotasViolated

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              apovzner Anna Povzner
              mjsax Matthias J. Sax
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: