Major Description
we can configure abfs and s3a to print out the filesystem IOStats in close(), but this doesn't let us see the state of long-lived processes, including latencies, error rates, and other important information.
Proposed: we add a means by which all cached filesystems which provide IOStatistics can log them to a file/console while the process continues to run
- a configurable scheduled thread which enumerates all open filesystem instances, gets their iostats and if non empty dumps to a log...log configuration can set that to the console or elsewhere. Good: can run without intervention; bad: one more thread to manage.
- option to add a signal handler which will do this whenever a SIGUSR2 is raised. JVM signal handling is a bit of a brittle feature, so it should be off by default.
org.apache.hadoop.service.launcher.IrqHandler supports signal wire-up. good: gives us an equivalent of kill -quit to get a view of iostats. bad: you need to be on the machine