[HADOOP-12659] Incorrect usage of config parameters in token manager of KMS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.1, 2.6.2
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: security
• Hadoop Flags: Reviewed
• Assignee: Mingliang Liu
• Reporter: Tianyin Xu
• Created: 19/Dec/15 02:13
• Updated: 30/Aug/16 01:20
• Resolved: 21/Jan/16 21:54

Description

Hi, the usage of the following configs of Key Management Server (KMS) are problematic:
hadoop.kms.authentication.delegation-token.renew-interval.sec
hadoop.kms.authentication.delegation-token.removal-scan-interval.sec

The name indicates that the units are sec, and the online doc shows that the default values are 86400 and 3600, respectively.
https://hadoop.apache.org/docs/stable/hadoop-kms/index.html
which is also defined in

DelegationTokenManager.java

 55   public static final String RENEW_INTERVAL = PREFIX + "renew-interval.sec";
 56   public static final long RENEW_INTERVAL_DEFAULT = 24 * 60 * 60;
 ...
 58   public static final String REMOVAL_SCAN_INTERVAL = PREFIX +
 59       "removal-scan-interval.sec";
 60   public static final long REMOVAL_SCAN_INTERVAL_DEFAULT = 60 * 60;

However, in DelegationTokenManager.java and ZKDelegationTokenSecretManager.java, these two parameters are used incorrectly.

1. DelegationTokenManager.java

 70           conf.getLong(RENEW_INTERVAL, RENEW_INTERVAL_DEFAULT) * 1000,
 71           conf.getLong(REMOVAL_SCAN_INTERVAL, 
 72               REMOVAL_SCAN_INTERVAL_DEFAULT * 1000));

Apparently, at Line 72, REMOVAL_SCAN_INTERVAL should be used in the same way as RENEW_INTERVAL, like

72c72
<               REMOVAL_SCAN_INTERVAL_DEFAULT * 1000));
---
>               REMOVAL_SCAN_INTERVAL_DEFAULT) * 1000);

Currently, the unit of hadoop.kms.authentication.delegation-token.removal-scan-interval.sec is not sec but millisec.

2. ZKDelegationTokenSecretManager.java

142         conf.getLong(DelegationTokenManager.RENEW_INTERVAL,
143             DelegationTokenManager.RENEW_INTERVAL_DEFAULT * 1000),
144         conf.getLong(DelegationTokenManager.REMOVAL_SCAN_INTERVAL,
145             DelegationTokenManager.REMOVAL_SCAN_INTERVAL_DEFAULT) * 1000);

The situation is the opposite in this class that hadoop.kms.authentication.delegation-token.renew-interval.sec is wrong but the other is correct...
A patch should be like

143c143
<             DelegationTokenManager.RENEW_INTERVAL_DEFAULT * 1000),
---
>             DelegationTokenManager.RENEW_INTERVAL_DEFAULT) * 1000,

Thanks!

Issue Links

• is broken by: HADOOP-11017 KMS delegation token secret manager should be able to use zookeeper as store
• relates to: HIVE-13236 LLAP: token renewal interval needs to be set