[SAMZA-2297] InMemorySystemAdmin offsets are off-by-one in some cases

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3
• Assignee: Cameron Lee
• Reporter: Cameron Lee
• Created: 08/Aug/19 19:13
• Updated: 26/Mar/20 05:37
• Resolved: 19/Aug/19 21:13

Description

1. When a stream is empty for an "InMemory" system, the admin returns that the oldest/newest offset is "0" and the upcoming offset is "1".
2. The "newest" offset returned by the admin is one higher than the offset of the last message in the list. Therefore, consuming from this offset would give no messages.
3. The "upcoming" offset returned by the admin is two higher than the offset of the last message in the list. However, the next message actually gets an offset which is (offset of the last message + 1).

For case (1), when using UPCOMING offsets, this results in missing the first message in a stream when the job starts while the stream is empty but then messages get produced to the stream. The current usages use the InMemorySystemDescriptor which uses OLDEST offsets, so this issue is not observed in the current tests. However, internal streams don't use the descriptor, so when trying to move some other tests to an in-memory system, those tests run into this problem.

For case (2)/(3), the current usage of the in-memory system doesn't seem to be impacted. It seems to be because the newest/upcoming offsets aren't used to consume in the current tests.

Issue Links

• relates to: SAMZA-2495 InMemorySystemAdmin offsets off by one when END_OF_STREAM present
• links to: GitHub Pull Request #1133