Details
-
Improvement
-
Status: Open
-
Normal
-
Resolution: Unresolved
-
None
-
None
Description
External profilers are pretty inadequate for getting accurate information at the granularity we're working at: tracing is too high overhead, so measures something completely different, and sampling suffers from bias of attribution due to the way the stack traces are retrieved. Hyperthreading can make this even worse.
I propose to introduce an extremely low overhead tracing feature that must be enabled with a system property that will trace operations within the node only, so that we can perform various accurate low level analyses of performance. This information will include threading info, so that we can trace hand off delays and actual active time spent processing an operation. With the property disabled there will be no increased burden of tracing, however I hope to keep the total trace burden to less than one microsecond, and any single trace command to a few tens of nanos.