Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1315

Add fsck event to audit log and remove other audit log events corresponding to FSCK listStatus and open calls

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: namenode, tools
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      When running fsck, audit log events are not logged for listStatus and open are not logged. A new event with cmd=fsck is logged with ugi field set to the user requesting fsck and src field set to the fsck path.

      Description

      In Yahoo production cluster audit logs more than 50% of entries comes from FSCK. FSCK makes same Namenode method calls as clients. Because of this, the operations are logged in audit logs similar to operations invoked by the client. I want to make a change to namenode not to log operations invoked by FSCK in audit logs.

      1. HDFS-1315.patch
        10 kB
        Suresh Srinivas
      2. HDFS-1315.1.patch
        18 kB
        Suresh Srinivas
      3. HDFS-1315.y20.patch
        18 kB
        Suresh Srinivas

        Issue Links

          Activity

          Suresh Srinivas created issue -
          Suresh Srinivas made changes -
          Field Original Value New Value
          Link This issue relates to HDFS-1316 [ HDFS-1316 ]
          Suresh Srinivas made changes -
          Summary Remove access log entries generated when FSCK calls listStatus and open methods Remove audit log entries generated when FSCK calls listStatus and open methods
          Description In Yahoo production cluster access logs more than 50% of entries comes from FSCK. FSCK makes same {{Namenode}} method calls as clients. Because of this, the operations are logged in access logs similar to operations invoked by the client. I want to make a change to namenode not to log operations invoked by FSCK in access logs. In Yahoo production cluster audit logs more than 50% of entries comes from FSCK. FSCK makes same {{Namenode}} method calls as clients. Because of this, the operations are logged in audit logs similar to operations invoked by the client. I want to make a change to namenode not to log operations invoked by FSCK in audit logs.
          Suresh Srinivas made changes -
          Attachment HDFS-1315.patch [ 12450243 ]
          Suresh Srinivas made changes -
          Attachment HDFS-1315.1.patch [ 12450280 ]
          Suresh Srinivas made changes -
          Attachment HDFS-1315.1.patch [ 12450287 ]
          Suresh Srinivas made changes -
          Attachment HDFS-1315.1.patch [ 12450280 ]
          Suresh Srinivas made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Suresh Srinivas made changes -
          Summary Remove audit log entries generated when FSCK calls listStatus and open methods Add fsck to audit log and remove other audit log corresponding to FSCK listStatus and open calls
          Suresh Srinivas made changes -
          Summary Add fsck to audit log and remove other audit log corresponding to FSCK listStatus and open calls Add fsck event to audit log and remove other audit log events corresponding to FSCK listStatus and open calls
          Suresh Srinivas made changes -
          Attachment HDFS-1315.y20.patch [ 12450338 ]
          Suresh Srinivas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change, Reviewed]
          Release Note When running fsck, audit log events are not logged for listStatus and open are not logged. A new event with cmd=fsck is logged with ugi field set to the user requesting fsck and src field set to the fsck path.
          Fix Version/s 0.22.0 [ 12314241 ]
          Resolution Fixed [ 1 ]
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Suresh Srinivas
              Reporter:
              Suresh Srinivas
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development