Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14346

Better time precision in getTimeDuration

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.4, 3.3.0, 3.2.1, 3.1.3
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, Configuration#getTimeDuration has the following signature:

        /**
         * Return time duration in the given time unit. Valid units are encoded in
         * properties as suffixes: nanoseconds (ns), microseconds (us), milliseconds
         * (ms), seconds (s), minutes (m), hours (h), and days (d).
         * @param name Property name
         * @param defaultValue Value returned if no mapping exists.
         * @param unit Unit to convert the stored property, if it exists.
         * @throws NumberFormatException If the property stripped of its unit is not
         *         a number
         */
        public long getTimeDuration(String name, long defaultValue, TimeUnit unit)
      

      This may lose precision in case the default time unit is larger than the time unit that the configuration value is converted to in the call sites of this method. For instance, in EditLogTailer this method is used in the following manner:

          logRollPeriodMs = conf.getTimeDuration(
              DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_KEY,
              DFSConfigKeys.DFS_HA_LOGROLL_PERIOD_DEFAULT, TimeUnit.SECONDS) * 1000;
      
          sleepTimeMs = conf.getTimeDuration(
              DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_KEY,
              DFSConfigKeys.DFS_HA_TAILEDITS_PERIOD_DEFAULT, TimeUnit.SECONDS) * 1000;
      

      in both cases, the default time unit is second, and the configuration value is converted into milli-seconds. Precision is lost when people want to specify sub-second time duration such as 100ms, which will be converted to 0ms.

        Attachments

        1. HDFS-14346.002.patch
          16 kB
          Chao Sun
        2. HDFS-14346.001.patch
          16 kB
          Chao Sun
        3. HDFS-14346.000.patch
          16 kB
          Chao Sun

          Issue Links

            Activity

              People

              • Assignee:
                csun Chao Sun
                Reporter:
                csun Chao Sun
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: