Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14509

DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x

    XMLWordPrintableJSON

Details

    Description

      According to the doc, if we want to upgrade cluster from 2.x to 3.x, we need upgrade NN first. And there will be a intermediate state that NN is 3.x and DN is 2.x. At that moment, if a client reads (or writes) a block, it will get a block token from NN and then deliver the token to DN who can verify the token. But the verification in the code now is :

      BlockTokenSecretManager.java
      public void checkAccess(...)
      {
          ...
          id.readFields(new DataInputStream(new ByteArrayInputStream(token.getIdentifier())));
          ...
          if (!Arrays.equals(retrievePassword(id), token.getPassword())) {
            throw new InvalidToken("Block token with " + id.toString()
                + " doesn't have the correct token password");
          }
      }
      

      And retrievePassword(id) is:

       
      public byte[] retrievePassword(BlockTokenIdentifier identifier)
      {
          ...
          return createPassword(identifier.getBytes(), key.getKey());
      }
      

      So, if NN's identifier add new fields, DN will lose the fields and compute wrong password.

      Attachments

        1. HDFS-14509-001.patch
          21 kB
          Brahma Reddy Battula
        2. HDFS-14509-002.patch
          5 kB
          Konstantin Shvachko
        3. HDFS-14509-003.patch
          5 kB
          Yuxuan Wang
        4. HDFS-14509-branch-2.001.patch
          5 kB
          Yuxuan Wang

        Issue Links

          Activity

            People

              John Smith Yuxuan Wang
              John Smith Yuxuan Wang
              Votes:
              0 Vote for this issue
              Watchers:
              24 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: