Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14441

LoadBalancingKMSClientProvider#addDelegationTokens should add delegation tokens from all KMS instances

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: kms
    • Labels:
      None
    • Environment:

      CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption

    • Target Version/s:

      Description

      LoadBalancingKMSClientProvider only gets delegation token from one KMS instance, in a round-robin fashion. This is arguably a bug, as JavaDoc for KeyProviderDelegationTokenExtension#addDelegationTokens states:

      /**

      • The implementer of this class will take a renewer and add all
      • delegation tokens associated with the renewer to the
      • <code>Credentials</code> object if it is not already present,
        ...

      **/

      This bug doesn't pop up very often, because HDFS clients such as MapReduce unintentionally calls FileSystem#addDelegationTokens multiple times.

      We have a custom client that accesses HDFS/KMS-HA using delegation token, and we were puzzled why it always throws "Failed to find any Kerberos tgt" exceptions talking to one KMS but not the other. Turns out that client couldn't talk to the KMS because FileSystem#addDelegationTokens only gets one KMS delegation token at a time.

        Attachments

        1. HADOOP-14441.branch-2.005.patch
          3 kB
          Wei-Chiu Chuang
        2. HADOOP-14441.004.patch
          13 kB
          Wei-Chiu Chuang
        3. HADOOP-14441.003.patch
          13 kB
          Wei-Chiu Chuang
        4. HADOOP-14441.002.patch
          12 kB
          Wei-Chiu Chuang
        5. HADOOP-14441.001.patch
          10 kB
          Wei-Chiu Chuang

          Issue Links

            Activity

              People

              • Assignee:
                weichiu Wei-Chiu Chuang
                Reporter:
                weichiu Wei-Chiu Chuang
              • Votes:
                1 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: