Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12362

Set hadoop.tmp.dir and hadoop.log.dir in pom

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      There are some compelling features in later version of surefire which lets one exclude/include tests based the content of a file, re-running of test case etc.

      However introduced in Surefire 2.18 is also https://issues.apache.org/jira/browse/SUREFIRE-649. Which changed the convention of null properties to empty string values (""). This only applies to forked tests such as the MapReduce tests and cause a couple of them to fail because of functionality that is directly or indirectly dependent on the value being null. One such example is Configuration.substituteVars() and TaskLog.getBaseLogDir().

      substituteVars() shows the issue when the getProperty returns empty String, skipping the getRaw(var) expression. One way to work around this could be

       
         if (val == null || val.isEmpty()) {
              String raw = getRaw(var);
              if (raw != null) {
                // raw contains a value, otherwise default to whatever System.getProperty returned
                // since it could be an empty string
                val = raw;
              }
            }
      

      getBaseLogDir, similarly when returns an empty string the schematics of java.io.File differs dependent on whether parent is null or "". A null value is interpreted as new File(file); whereas "" will be interpreted as new File(defaultParent /* / */, file);

      This could simply be addressed with

        static String getBaseLogDir() {
          String logDir = System.getProperty("hadoop.log.dir");
          // there is a difference how null and "" is treated as a parent
          // directory when creating a file
          return logDir == null || logDir.isEmpty() ? null : logDir;
        }
      

        Attachments

        1. mr-6455.1.patch
          2 kB
          Charlie Helin
        2. mr-6455.2.patch
          2 kB
          Karthik Kambatla
        3. mr-6455.2.patch
          2 kB
          Charlie Helin
        4. mr-6455.3.patch
          2 kB
          Charlie Helin
        5. mr-6455.4.patch
          2 kB
          Charlie Helin

          Activity

            People

            • Assignee:
              chelin Charlie Helin
              Reporter:
              chelin Charlie Helin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: