Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In some situations having a config to turn on/off observer reads dynamically may be useful. For instance, some applications may be sensitive for data freshness and want to always reach directly to active NN. In a more complex scenario, services such as Presto may want to apply observer reads for different types of queries. In this case, simply change dfs.client.failover.proxy.provider.<nameservice> may not be enough, since with FileSystem cache (which is usually turned on) will ignore the change and still use the same FileSystem object.
Here I'm proposing to add a flag in HdfsClientConfigKeys, such as dfs.client.observer.reads.enabled, that can be used to dynamically turn on/off observer reads. The FileSystem cache key should also take account of this flag in its hashCode and equals impl, so that different FileSystem objects will be used depending on the flag.
cc shv, xkrogen, vagarychen, zero45 for discussion.
Attachments
Issue Links
- links to
