Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0, 1.1.4
    • Component/s: None
    • Labels:
      None

      Description

      HadoopFileSystem logging is very noisy when trying to get the Hadoop configuration, which happens for every HadoopFileSystem. This happens for example when a checkpoint is written to HDFS in case of HA.

      1) Two log messages relate to the user config:

      final String hdfsDefaultPath = GlobalConfiguration.getString(ConfigConstants.HDFS_DEFAULT_CONFIG, null);
      		if (hdfsDefaultPath != null) {
      			retConf.addResource(new org.apache.hadoop.fs.Path(hdfsDefaultPath));
      		} else {
      			LOG.debug("Cannot find hdfs-default configuration file");
      		}
      

      Since this is reflected in the logged configuration properties, it is mostly unnecessary to log over and over again. Therefore I propose to go with TRACE.

      (2) Two log messages relate to the default config files of Hadoop. There I would change the debug message to happen only when they are not found. If it is still too noisy, we could completely remove it or decrease to TRACE at another time.

        Activity

        Hide
        uce Ufuk Celebi added a comment -

        Fixed in bf859e7 (master), 28e44e7 (release-1.1).

        Show
        uce Ufuk Celebi added a comment - Fixed in bf859e7 (master), 28e44e7 (release-1.1).

          People

          • Assignee:
            Unassigned
            Reporter:
            uce Ufuk Celebi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development