[HDFS-14509] DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 2.10.0, 3.3.0, 3.1.4, 3.2.2
• Labels: release-blocker
• Target Version/s: 2.10.0, 3.3.0, 3.1.3, 3.2.2
• Assignee: Yuxuan Wang
• Reporter: Yuxuan Wang
• Created: 23/May/19 12:42
• Updated: 16/Sep/23 12:30
• Resolved: 10/Oct/19 20:31

Description

According to the doc, if we want to upgrade cluster from 2.x to 3.x, we need upgrade NN first. And there will be a intermediate state that NN is 3.x and DN is 2.x. At that moment, if a client reads (or writes) a block, it will get a block token from NN and then deliver the token to DN who can verify the token. But the verification in the code now is :

BlockTokenSecretManager.java

public void checkAccess(...)
{
    ...
    id.readFields(new DataInputStream(new ByteArrayInputStream(token.getIdentifier())));
    ...
    if (!Arrays.equals(retrievePassword(id), token.getPassword())) {
      throw new InvalidToken("Block token with " + id.toString()
          + " doesn't have the correct token password");
    }
}

And retrievePassword(id) is:

 
public byte[] retrievePassword(BlockTokenIdentifier identifier)
{
    ...
    return createPassword(identifier.getBytes(), key.getKey());
}

So, if NN's identifier add new fields, DN will lose the fields and compute wrong password.

Issue Links

• is related to: HDFS-13596 NN restart fails after RollingUpgrade from 2.x to 3.x
• links to: GitHub Pull Request #1129