[HDFS-16252] Correct docs for dfs.http.client.retry.policy.spec

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.2
• Fix Version/s: 3.4.0, 3.3.2
• Component/s: documentation
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Stephen O'Donnell
• Reporter: Stephen O'Donnell
• Created: 04/Oct/21 10:40
• Updated: 27/Jan/24 02:34
• Resolved: 06/Oct/21 16:26

Description

The hdfs-default doc for dfs.http.client.retry.policy.spec is incorrect, as it has the wait time and retries switched around in the descriptio. Also, the doc for dfs.client.retry.policy.spec is not present and should be the same as for dfs.http.client.retry.policy.spec.

The code shows the timeout is first and then the number of retries:

    String  POLICY_SPEC_KEY = PREFIX + "policy.spec";
    String  POLICY_SPEC_DEFAULT = "10000,6,60000,10"; //t1,n1,t2,n2,...


    // In RetryPolicies.java, we can see it gets the timeout as the first in the pair


   /**
     * Parse the given string as a MultipleLinearRandomRetry object.
     * The format of the string is "t_1, n_1, t_2, n_2, ...",
     * where t_i and n_i are the i-th pair of sleep time and number of retries.
     * Note that the white spaces in the string are ignored.
     *
     * @return the parsed object, or null if the parsing fails.
     */
    public static MultipleLinearRandomRetry parseCommaSeparatedString(String s) {
      final String[] elements = s.split(",");
      if (elements.length == 0) {
        LOG.warn("Illegal value: there is no element in \"" + s + "\".");
        return null;
      }
      if (elements.length % 2 != 0) {
        LOG.warn("Illegal value: the number of elements in \"" + s + "\" is "
            + elements.length + " but an even number of elements is expected.");
        return null;
      }

      final List<RetryPolicies.MultipleLinearRandomRetry.Pair> pairs
          = new ArrayList<RetryPolicies.MultipleLinearRandomRetry.Pair>();
   
      for(int i = 0; i < elements.length; ) {
        //parse the i-th sleep-time
        final int sleep = parsePositiveInt(elements, i++, s);
        if (sleep == -1) {
          return null; //parse fails
        }

        //parse the i-th number-of-retries
        final int retries = parsePositiveInt(elements, i++, s);
        if (retries == -1) {
          return null; //parse fails
        }

        pairs.add(new RetryPolicies.MultipleLinearRandomRetry.Pair(retries, sleep));
      }
      return new RetryPolicies.MultipleLinearRandomRetry(pairs);
  }

This change simply updates the docs.