[KAFKA-13007] KafkaAdminClient getListOffsetsCalls builds cluster snapshot for every topic partition

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.8.1, 3.0.0
• Component/s: clients
• Assignee: Jeff Kim
• Reporter: Jeff Kim
• Created: 29/Jun/21 06:17
• Updated: 01/Jul/21 21:17
• Resolved: 01/Jul/21 21:17

Description

From KafkaAdminClient#getListOffsetsCalls

```

for (Map.Entry<TopicPartition, OffsetSpec> entry: topicPartitionOffsets.entrySet()) {

...

Node node = mr.cluster().leaderFor(tp);

```

here we build the cluster snapshot for each topic partition. instead, we should reuse a snapshot. this will reduce the time complexity from O( n^2 ) to O( n ).

for manual testing (used AK 2.8), i've passed in a map of 6K topic partitions to listOffsets

without snapshot reuse:
duration of building futures from metadata response: 15582 milliseconds
total duration of listOffsets: 15743 milliseconds

with reuse:
duration of building futures from metadata response: 24 milliseconds
total duration of listOffsets: 235 milliseconds

Affects all versions since Admin & KafkaAdminClient introduced listOffsets (original PR: https://github.com/apache/kafka/pull/7296)

Issue Links

• links to: GitHub Pull Request #10940