Details
-
Improvement
-
Status: Patch Available
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
-
Ubuntu LXC
Description
Hadoop configuration parsing re-parses all configuration files when a new resource is added to the configuration object.
This is wasteful and runs through every deprecation check unnecessarily.
JobConf clone = new JobConf(job);
clone.addResource(...);
will reparse every file including core-site, hdfs-site etc.
Resource addition can be taken care of cleanly, if the addResourceObject() call applies the new resource, followed by applying the overlay, without re-parsing any of the older files already loaded into the hashtable.
Attachments
Attachments
Issue Links
- is related to
-
HIVE-4486 FetchOperator slows down SMB map joins by 50% when there are many partitions
- Closed