[KAFKA-7794] kafka.tools.GetOffsetShell does not return the offset in some cases

• Type: Bug
• Status: In Progress
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 0.10.2.0, 0.10.2.1, 0.10.2.2
• Component/s: tools
• Labels: Kafka ShellCommands kafka-0.10 offset shell shell-script shellscript tools usability
• Assignee: Kartik
• Reporter: Daniele Ascione
• Created: 08/Jan/19 17:32
• Updated: 16/Feb/19 16:54

Description

For some input for the timestamps (different from -1 or -2) the GetOffset is not able to retrieve the offset.

For example, if x is the timestamp in that "not working range", and you execute:

bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --time x

The output is:

MY_TOPIC:8:
MY_TOPIC:2:
MY_TOPIC:5:
MY_TOPIC:4:
MY_TOPIC:7:
MY_TOPIC:1:
MY_TOPIC:9:

while after the last ":" an integer representing the offset is expected.

---

Steps to reproduce it:

1. Consume all the messages from the beginning and print the timestamp:

   bin/kafka-simple-consumer-shell.sh --no-wait-at-logend --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --property print.timestamp=true  > messages

1. Sort the messages by timestamp and get some of the oldest messages:

    awk -F "CreateTime:" '{ print $2}' messages | sort -n > msg_sorted

1. Take (for example) the timestamp of the 10th oldest message, and see if GetOffsetShell is not able to print the offset:

   timestamp="$(sed '10q;d' msg_sorted | cut -f1)"
   bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --time $timestamp
   # The output should be something like:
   # MY_TOPIC:1:
   # MY_TOPIC:2:
   (repeated for every partition)

1. Verify that the message with that timestamp is still in Kafka:

   bin/kafka-simple-consumer-shell.sh --no-wait-at-logend --broker-list $KAFKA_ADDRESS --topic $MY_TOPIC --property print.timestamp=true | grep "CreateTime:$timestamp"