Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3501

Checkpointing with security enabled will stop working after ticket lifetime expires

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: ha, namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The StandbyCheckpointer and 2NN currently do the right thing in renewing their krb5 creds before attempting to perform a checkpoint to the active NN, but the active NN makes no attempt to renew its own krb5 creds before connecting to the standby NN or 2NN to fetch the new merged fsimage file.

      1. HDFS-3501.patch
        1 kB
        Aaron T. Myers

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1101 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1101/)
        HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1101 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1101/ ) HDFS-3501 . Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1067 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1067/)
        HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1067 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1067/ ) HDFS-3501 . Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2338 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2338/)
        HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2338 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2338/ ) HDFS-3501 . Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2392 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2392/)
        HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2392 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2392/ ) HDFS-3501 . Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2320 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2320/)
        HDFS-3501. Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2320 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2320/ ) HDFS-3501 . Checkpointing with security enabled will stop working after ticket lifetime expires. Contributed by Aaron T. Myers (Revision 1346106) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346106 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java
        Hide
        Eli Collins added a comment -

        This is a 1-liner and only runs when security is enabled so new test case won't cover it.

        I've committed this and merged to branch-2. Thanks ATM.

        Show
        Eli Collins added a comment - This is a 1-liner and only runs when security is enabled so new test case won't cover it. I've committed this and merged to branch-2. Thanks ATM.
        Hide
        Eli Collins added a comment -

        +1 looks good

        ATM mentioned offline that he tested doing checkpoints every 30s for 15 mins with a lowered ticket renewal.

        Show
        Eli Collins added a comment - +1 looks good ATM mentioned offline that he tested doing checkpoints every 30s for 15 mins with a lowered ticket renewal.
        Hide
        Aaron T. Myers added a comment -

        Here's a patch which fixes the issue. I tested this manually in a secure+HA setup and confirmed that it works as expected.

        Show
        Aaron T. Myers added a comment - Here's a patch which fixes the issue. I tested this manually in a secure+HA setup and confirmed that it works as expected.

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development