Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11223

Offer a read-only conf alternative to new Configuration()

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
    • Target Version/s:

      Description

      new Configuration() is called from several static blocks across Hadoop.

      This is incredibly inefficient, since each one of those involves primarily XML parsing at a point where the JIT won't be triggered & interpreter mode is essentially forced on the JVM.

      The alternate solution would be to offer a Configuration::getDefault() alternative which disallows any modifications.

      At the very least, such a method would need to be called from

      1. org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
      2. org.apache.hadoop.security.SecurityUtil::<clinit>()
      3. org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>

        Attachments

        1. HADOOP-11223.001.patch
          1 kB
          Varun Saxena
        2. HADOOP-11223.002.patch
          21 kB
          Michael Miller
        3. HADOOP-11223.003.patch
          21 kB
          Michael Miller
        4. HADOOP-11223.004.patch
          23 kB
          Michael Miller
        5. HADOOP-11223.005.patch
          23 kB
          Michael Miller

          Issue Links

            Activity

              People

              • Assignee:
                milleruntime Michael Miller
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                3 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated: