Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
For large instances (i.e. lots of clients for a given tserver) we create oodles of threads on the tserver. This makes for difficulty in predicting performance, memory usage, etc. Moreover, we have operations that recurse, like a server querying itself, that we currently solve by having separate thread pools for regular table operations and metadata table operations, and we "disallow" things like an iterator writing to another table. One alternative option would be to switch to a Proactor pattern: https://en.wikipedia.org/wiki/Proactor_pattern
The core of this would be to switch to using a selection set rather than a thread per active connection, and then wrap everything in sessions that make progress in something like a state model, with states that account for asynchronous communications and remote work.