Hadoop Common
  1. Hadoop Common
  2. HADOOP-8552

Conflict: Same security.log.file for multiple users.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 2.0.0-alpha
    • Fix Version/s: 1.1.0, 2.0.1-alpha
    • Component/s: conf, security
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. In the presence of multiple users, this can lead to a potential conflict.

      Adding username to the log file would avoid this scenario.

      1. HADOOP-8552_branch1.patch
        1 kB
        Karthik Kambatla
      2. HADOOP-8552_branch2.patch
        2 kB
        Karthik Kambatla

        Activity

        Hide
        Suresh Srinivas added a comment -

        Usename is in the log entries right. Can you describe the problem better?

        Show
        Suresh Srinivas added a comment - Usename is in the log entries right. Can you describe the problem better?
        Hide
        Karthik Kambatla (Inactive) added a comment -

        Hi Suresh,

        Thanks for looking into this.

        The problem we came across was – at times, multiple (2 in our case) users might write to the same file (at least attempt to open the file) simultaneously, because the hadoop.security.log.file is set to the same value.

        Please let me know if I am missing something here.

        Thanks

        Show
        Karthik Kambatla (Inactive) added a comment - Hi Suresh, Thanks for looking into this. The problem we came across was – at times, multiple (2 in our case) users might write to the same file (at least attempt to open the file) simultaneously, because the hadoop.security.log.file is set to the same value. Please let me know if I am missing something here. Thanks
        Hide
        Karthik Kambatla (Inactive) added a comment -

        I am uploading patches for branch-1 and branch-2.

        Show
        Karthik Kambatla (Inactive) added a comment - I am uploading patches for branch-1 and branch-2.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535000/HADOOP-8552_branch2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1180//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535000/HADOOP-8552_branch2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1180//console This message is automatically generated.
        Hide
        Karthik Kambatla (Inactive) added a comment -

        Updating the patch after testing.

        Tested it on a secure cluster, and the appropriate log file is created.

        Show
        Karthik Kambatla (Inactive) added a comment - Updating the patch after testing. Tested it on a secure cluster, and the appropriate log file is created.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12536148/HADOOP-8552_branch1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1194//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536148/HADOOP-8552_branch1.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1194//console This message is automatically generated.
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Devaraj Das added a comment -

        Hi Karthik, is this on the client or on the server side? (Guessing its on client.. please confirm). In general, the audit log stuff doesn't make sense on the client side. It's meant to be used on the server side only (and in deployments I know about, the security audit logging is turned off on the client side).
        Your patch will work though. But I'll note that it might be introducing compatibility issues due to the filename change of the log file (if someone is collecting logs based on file names, etc.).

        Show
        Devaraj Das added a comment - Hi Karthik, is this on the client or on the server side? (Guessing its on client.. please confirm). In general, the audit log stuff doesn't make sense on the client side. It's meant to be used on the server side only (and in deployments I know about, the security audit logging is turned off on the client side). Your patch will work though. But I'll note that it might be introducing compatibility issues due to the filename change of the log file (if someone is collecting logs based on file names, etc.).
        Hide
        Karthik Kambatla (Inactive) added a comment -

        Devaraj, thanks for the feedback.

        It is both on the client/server side. By server side, I mean for the jobtracker/namenode. Thanks for pointing the potential compatibility issue, I agree we need to note the incompatibility in log file change.

        Show
        Karthik Kambatla (Inactive) added a comment - Devaraj, thanks for the feedback. It is both on the client/server side. By server side, I mean for the jobtracker/namenode. Thanks for pointing the potential compatibility issue, I agree we need to note the incompatibility in log file change.
        Hide
        Alejandro Abdelnur added a comment -

        Devaraj, are you OK with this patch after Karthik's clarifications?

        Show
        Alejandro Abdelnur added a comment - Devaraj, are you OK with this patch after Karthik's clarifications?
        Hide
        Devaraj Das added a comment -

        Yes.

        Show
        Devaraj Das added a comment - Yes.
        Hide
        Alejandro Abdelnur added a comment -

        Thanks Karthik. Committed to trunk, branch-1 and branch-2.

        Show
        Alejandro Abdelnur added a comment - Thanks Karthik. Committed to trunk, branch-1 and branch-2.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2545/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2545/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2480 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2480/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2480 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2480/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2500 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2500/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2500 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2500/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1106/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1106/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1139 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1139/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1139 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1139/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        Suresh Srinivas added a comment -

        Alejandro, when committing incompatible changes, could you please add the change description in CHANGES.txt under INCOMPATIBLE CHANGES section. Also could you please add release notes on what is incompatible here and how to get around it.

        Show
        Suresh Srinivas added a comment - Alejandro, when committing incompatible changes, could you please add the change description in CHANGES.txt under INCOMPATIBLE CHANGES section. Also could you please add release notes on what is incompatible here and how to get around it.
        Hide
        Suresh Srinivas added a comment -

        I also added this change in CHANGES.txt in branch 1.1.

        Show
        Suresh Srinivas added a comment - I also added this change in CHANGES.txt in branch 1.1.
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop-1.1.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

          People

          • Assignee:
            Karthik Kambatla
            Reporter:
            Karthik Kambatla
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development