Hadoop Common
  1. Hadoop Common
  2. HADOOP-6693

Add metrics to track kerberos login activity

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      New metrics "login" of type MetricTimeVaryingRate is added under new metrics context name "ugi" and metrics record name "ugi".

      Description

      Need metrics to track kerberos login activity such as login rate and the time taken for login.

      1. HADOOP-6693.patch
        9 kB
        Suresh Srinivas
      2. HADOOP-6693.rel20.1.patch
        10 kB
        Suresh Srinivas
      3. HADOOP-6693.rel20.patch
        10 kB
        Suresh Srinivas

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        4d 17h 35m 2 Suresh Srinivas 17/Aug/10 16:41
        Open Open Patch Available Patch Available
        126d 21m 3 Suresh Srinivas 17/Aug/10 16:42
        Patch Available Patch Available Resolved Resolved
        4h 23m 1 Suresh Srinivas 17/Aug/10 21:05
        Resolved Resolved Closed Closed
        481d 10h 13m 1 Konstantin Shvachko 12/Dec/11 06:19
        Konstantin Shvachko made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk #427 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/427/)
        HADOOP-6693. Add metrics to track kerberol login activity. Contributed by Suresh Srinivas.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk #427 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/427/ ) HADOOP-6693 . Add metrics to track kerberol login activity. Contributed by Suresh Srinivas.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #359 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/359/)
        HADOOP-6693. Add metrics to track kerberol login activity. Contributed by Suresh Srinivas.

        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #359 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/359/ ) HADOOP-6693 . Add metrics to track kerberol login activity. Contributed by Suresh Srinivas.
        Suresh Srinivas made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Release Note New metrics "login" of type MetricTimeVaryingRate is added under new metrics context name "ugi" and metrics record name "ugi".
        Resolution Fixed [ 1 ]
        Hide
        Suresh Srinivas added a comment -

        I committed the patch.

        Show
        Suresh Srinivas added a comment - I committed the patch.
        Hide
        Suresh Srinivas added a comment -

        Since hudson has some issues, ran the unit tests and testpatch. All the tests passed.

        Show
        Suresh Srinivas added a comment - Since hudson has some issues, ran the unit tests and testpatch. All the tests passed.
        Suresh Srinivas made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Suresh Srinivas made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Suresh Srinivas made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Suresh Srinivas made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Jitendra Nath Pandey added a comment -

        +1.
        The kinit login in spawnAutoRenewalThreadForUserCreds is not tracked by these metrics, which is fine because this thread is spawned only at clients and purpose of these metrics is to track logins on the servers.

        Show
        Jitendra Nath Pandey added a comment - +1. The kinit login in spawnAutoRenewalThreadForUserCreds is not tracked by these metrics, which is fine because this thread is spawned only at clients and purpose of these metrics is to track logins on the servers.
        Suresh Srinivas made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.patch [ 12451809 ]
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.1.patch [ 12451937 ]
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.patch [ 12451955 ]
        Hide
        Suresh Srinivas added a comment -

        New patch to fix a patch merge problem

        Show
        Suresh Srinivas added a comment - New patch to fix a patch merge problem
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.1.patch [ 12451937 ]
        Hide
        Suresh Srinivas added a comment -

        New trunk patch that incorporates additional change from HADOOP-6808.

        Show
        Suresh Srinivas added a comment - New trunk patch that incorporates additional change from HADOOP-6808 .
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.1.patch [ 12443016 ]
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.patch [ 12441216 ]
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.patch [ 12451809 ]
        Hide
        Suresh Srinivas added a comment -

        Trunk version of the patch.

        Show
        Suresh Srinivas added a comment - Trunk version of the patch.
        Hide
        Erik Steffl added a comment -

        If/when there is a patch for trunk the patch in https://issues.apache.org/jira/browse/HADOOP-6808 should also be included.

        Show
        Erik Steffl added a comment - If/when there is a patch for trunk the patch in https://issues.apache.org/jira/browse/HADOOP-6808 should also be included.
        Hide
        Owen O'Malley added a comment -

        +1

        Show
        Owen O'Malley added a comment - +1
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.rel20.1.patch [ 12443326 ]
        Hide
        Suresh Srinivas added a comment -

        New patch with unnecessary debug logs removed. Tested it on cluster to ensure metrics is being updated.

        Show
        Suresh Srinivas added a comment - New patch with unnecessary debug logs removed. Tested it on cluster to ensure metrics is being updated.
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.rel20.patch [ 12443325 ]
        Hide
        Suresh Srinivas added a comment -

        20 version of the patch. I added bumping up metrics in loginUserFromKeytabAndReturnUGI also.

        The login code in UserGroupInformation is repeated in many places. There is an opportunity to improve the code by making a method for it.

        Show
        Suresh Srinivas added a comment - 20 version of the patch. I added bumping up metrics in loginUserFromKeytabAndReturnUGI also. The login code in UserGroupInformation is repeated in many places. There is an opportunity to improve the code by making a method for it.
        Hide
        Owen O'Malley added a comment -

        +1

        Show
        Owen O'Malley added a comment - +1
        Suresh Srinivas made changes -
        Attachment HADOOP-6693.1.patch [ 12443016 ]
        Hide
        Suresh Srinivas added a comment -

        New patch adds MetricsTimeVaryingRate metrics for login success and failure.

        Show
        Suresh Srinivas added a comment - New patch adds MetricsTimeVaryingRate metrics for login success and failure.
        Hide
        Suresh Srinivas added a comment -

        Also should we add metrics for logout?

        Show
        Suresh Srinivas added a comment - Also should we add metrics for logout?
        Hide
        Suresh Srinivas added a comment -

        Owen, MetricsTimeVaryingRate exposes two pieces of information: Number of operations and average time taken for the operation. Does it cover your requirement?

        I will change the patch to include two metrics - one for tracking login success and another for login failure.

        Show
        Suresh Srinivas added a comment - Owen, MetricsTimeVaryingRate exposes two pieces of information: Number of operations and average time taken for the operation. Does it cover your requirement? I will change the patch to include two metrics - one for tracking login success and another for login failure.
        Hide
        Owen O'Malley added a comment -

        To figure out the time take for login, we also need the number of logins to divide by. I'd suggest adding the number of successes and failures for keytab logins.

        Show
        Owen O'Malley added a comment - To figure out the time take for login, we also need the number of logins to divide by. I'd suggest adding the number of successes and failures for keytab logins.
        Hide
        Suresh Srinivas added a comment -

        Note that the new metrics is aggregated across all the nodes in the cluster, since no distinction made in metrics record name about the node where the metrics was reported from.

        Show
        Suresh Srinivas added a comment - Note that the new metrics is aggregated across all the nodes in the cluster, since no distinction made in metrics record name about the node where the metrics was reported from.
        Suresh Srinivas made changes -
        Field Original Value New Value
        Attachment HADOOP-6693.patch [ 12441216 ]
        Hide
        Suresh Srinivas added a comment -

        Patch adds new metrics "login" of type MetricTimeVaryingRate under metrics context "ugi" and metrics record "ugi".

        Show
        Suresh Srinivas added a comment - Patch adds new metrics "login" of type MetricTimeVaryingRate under metrics context "ugi" and metrics record "ugi".
        Suresh Srinivas created issue -

          People

          • Assignee:
            Suresh Srinivas
            Reporter:
            Suresh Srinivas
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development