Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9570

Configuration.addResource() should only parse the new resource

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None
    • Environment:

      Ubuntu LXC

      Description

      Hadoop configuration parsing re-parses all configuration files when a new resource is added to the configuration object.

      This is wasteful and runs through every deprecation check unnecessarily.

      JobConf clone = new JobConf(job);
      clone.addResource(...);
      

      will reparse every file including core-site, hdfs-site etc.

      Resource addition can be taken care of cleanly, if the addResourceObject() call applies the new resource, followed by applying the overlay, without re-parsing any of the older files already loaded into the hashtable.

        Attachments

        1. HADOOP-9750.patch
          2 kB
          Gopal V
        2. HADOOP-9570.branch-2.8.patch
          2 kB
          Daryn Sharp
        3. HADOOP-9570.branch-2.8.001.patch
          2 kB
          Daryn Sharp

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                gopalv Gopal V
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: