[CRUNCH-630] KafkaRecordReader keeps retrying to poll data when the offset is reset to latest offset - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.15.0
Component/s: None
Labels:
None

Description

We recently saw this behavior where, if the offset it is trying to read from doesn't exist on Kafka due to retention policy, the offset gets reset to latest(default) and the KafkaRecordReader keeps retrying beyond KAFKA_EMPTY_RETRY_ATTEMPTS_KEY

...crunch.kafka.inputformat.KafkaRecordReader: No records retrieved but pending offsets to consume therefore polling again. Attempt 17/10

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

CRUNCH-630.patch
03/Jan/17 17:43
5 kB
Micah Whitacre

Activity

People

Assignee:: Micah Whitacre

Reporter:: Pooja Dhondge

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 12/Dec/16 15:39

Updated:: 26/Feb/17 18:33

Resolved:: 12/Jan/17 02:04