Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
0.9.2
-
None
-
None
Description
FLUME-155 is a good first cut but more improvements can be done. This jira is the place holder for some of these issues.
Feedback from Vibhor on first cut. (FLUME-155)
""""
Most of it looks good and very interesting. I'll give the high level comments here, gave minor comments on paper to Jon.
1) Missing the main Threading detail, it should be explained that there is a main driver thread, and different decorators can create their own threads too.
2) In the section talking about the sinks with buffered memories it says close() from a closing state "should blocks until resources are freed, it should attempt to flush its buffers before returning. For example, so if a network source has some buffered data, the network connection should be closed to prevent new data from entering, and then the buferred data should be flushed",
but at the same time it is said that the next() should keep on pulling data from the buffer even in the closing state. As both these methods (close() and next()) can be from different threads, it should be clearly mentioned how are we handling the race conditions while operating on the same data.
3) In both Source/Sink descriptions, it is said that if a timeout happens in a closing state, then we move to a close state. But do we ensure that we have given up all the resources before it? Else there will be an issue when someone later will try to open this decorator.
4) It should be clearly explained why and how we handle the Thread-Interrupted-Exception when a append() on sink gets interrupted while it is blocked.
"""
Attachments
Issue Links
- is blocked by
-
FLUME-155 Formalise behaviour of sinks, sources and decorators
-
- Closed
-