Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13777

Trim configuration values in `rumen`

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
      None

      Description

      The current implementation of ClassName.java in rumen does not follow the practice of trimming configuration values. This leads to silent and hard-to-diagnosis errors if users set values containing space or newline---basically classes supposed to need anonymization will not do.

      See the previous commits as reference (just list a few):
      HADOOP-6578. Configuration should trim whitespace around a lot of value types
      HADOOP-6534. Trim whitespace from directory lists initializing
      Patch is available against trunk
      HDFS-9708. FSNamesystem.initAuditLoggers() doesn't trim classnames
      HDFS-2799. Trim fs.checkpoint.dir values.
      YARN-3395. FairScheduler: Trim whitespaces when using username for queuename.
      YARN-2869. CapacityScheduler should trim sub queue names when parse configuration.

      Patch is available against trunk (tested):

      ClassName.java
      @@ -43,15 +43,13 @@ protected String getPrefix() {
      
         @Override
         protected boolean needsAnonymization(Configuration conf) {
      -    String[] preserves = conf.getStrings(CLASSNAME_PRESERVE_CONFIG);
      -    if (preserves != null) {
      -      // do a simple starts with check
      -      for (String p : preserves) {
      -        if (className.startsWith(p)) {
      -          return false;
      -        }
      +    String[] preserves = conf.getTrimmedStrings(CLASSNAME_PRESERVE_CONFIG);
      +    // do a simple starts with check
      +    for (String p : preserves) {
      +      if (className.startsWith(p)) {
      +        return false;
             }
           }
           return true;
         }
      

      (the NULL check is no longer needed because getTrimmedStrings returns an empty array if nothing is set)

        Attachments

        1. HADOOP-13777.0000.patch
          1 kB
          Tianyin Xu

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tianyin Tianyin Xu
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: