Details
-
Task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
Although sinks, sources et al have pretty simple APIs, the state diagram for how they should behave in response to these methods is left undefined.
For example:
1. Can open(..) block?
2. What should the behaviour be if open(...) is called twice in succession without an intervening close?
3. What if next(..) is called after close(..)? or before open(...)?
4. What should the policy be if a source has 'pending' events and close(..) is called?
There are many more similar questions. Some of them are already answered by convention in the code, where others are left unspecified. It should be pretty easy to formalise the answers, maybe via a state machine diagram (where the transitions are method calls).
There have been a lot of bugs inside Flume due to misbehaving sources or sinks, but given no formal specification to adhere to there was no way they could have been known to be buggy. Once we figure out the exact semantics of each call, we can write a test suite that puts every source and sink through its paces, and validate its behaviour, making it easier to find bugs.
Attachments
Issue Links
- blocks
-
FLUME-130 TestDiskFailoverAgent.testActiveE2EClose fails intermittently
- Resolved
-
FLUME-174 Create standard unit tests for decorators
- Resolved
-
FLUME-172 Create standard unit tests for sources
- Closed
-
FLUME-173 Create standard unit tests for Sinks
- Closed
-
FLUME-122 Make all Source/Sink/Decos API calls cancellable by throwing InterruptedExceptions
- Resolved
-
FLUME-238 Add more details in source/sink/decorator semantics
- Resolved
- is related to
-
FLUME-153 Nodes get LOST on config change
- Closed
- relates to
-
FLUME-101 update plugin docs and provide a working example
- Closed
-
FLUME-86 Clean up and document Source/Sink/Decorator plug-in mechanism
- Resolved