Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
fs, ha, hdfs-client
-
None
Description
The memory footprint of #DFSClient is very considerable in some special scenario since there are many #Configuration instances and occupy much memory resource (In an extreme case, org.apache.hadoop.conf.Configuration occupies over 600MB we meet under HDFS Federation an HA with QJM and there are dozens of NameNodes). I think some new Configuration instance is not necessary. Such as #ConfiguredFailoverProxyProvider initialization.
public ConfiguredFailoverProxyProvider(Configuration conf, URI uri, Class<T> xface, HAProxyFactory<T> factory) { this.xface = xface; this.conf = new Configuration(conf); ...... }
upload patch v1 for trunk and pending jenkins.