Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4466

Remove the deadlock from AbstractDelegationTokenSecretManager

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0
    • Fix Version/s: 1.2.0
    • Component/s: namenode, security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS-3374, new synchronization in AbstractDelegationTokenSecretManager.ExpiredTokenRemover was added to make sure the ExpiredTokenRemover thread can be interrupted in time. Otherwise TestDelegation fails intermittently because the MiniDFScluster thread could be shut down before tokenRemover thread.
      However, as Todd pointed out in HDFS-3374, a potential deadlock was introduced by its patch:

      • FSNamesystem.saveNamespace (holding FSN lock) calls DTSM.saveSecretManagerState (which takes DTSM lock)
      • ExpiredTokenRemover.run (holding DTSM lock) calls rollMasterKey calls updateCurrentKey calls logUpdateMasterKey which takes FSN lock
        So if there is a concurrent saveNamespace at the same tie as the expired token remover runs, it might make the NN deadlock.

      This JIRA is to track the change of removing the possible deadlock from AbstractDelegationTokenSecretManager.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                brandonli Brandon Li
                Reporter:
                brandonli Brandon Li
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: