Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8552

Conflict: Same security.log.file for multiple users.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 2.0.0-alpha
    • Fix Version/s: 1.1.0, 2.0.1-alpha
    • Component/s: conf, security
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      In log4j.properties, hadoop.security.log.file is set to SecurityAuth.audit. In the presence of multiple users, this can lead to a potential conflict.

      Adding username to the log file would avoid this scenario.

      1. HADOOP-8552_branch1.patch
        1 kB
        Karthik Kambatla
      2. HADOOP-8552_branch2.patch
        2 kB
        Karthik Kambatla

        Activity

        Hide
        sureshms Suresh Srinivas added a comment -

        Usename is in the log entries right. Can you describe the problem better?

        Show
        sureshms Suresh Srinivas added a comment - Usename is in the log entries right. Can you describe the problem better?
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        Hi Suresh,

        Thanks for looking into this.

        The problem we came across was – at times, multiple (2 in our case) users might write to the same file (at least attempt to open the file) simultaneously, because the hadoop.security.log.file is set to the same value.

        Please let me know if I am missing something here.

        Thanks

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - Hi Suresh, Thanks for looking into this. The problem we came across was – at times, multiple (2 in our case) users might write to the same file (at least attempt to open the file) simultaneously, because the hadoop.security.log.file is set to the same value. Please let me know if I am missing something here. Thanks
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        I am uploading patches for branch-1 and branch-2.

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - I am uploading patches for branch-1 and branch-2.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535000/HADOOP-8552_branch2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1180//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535000/HADOOP-8552_branch2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1180//console This message is automatically generated.
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        Updating the patch after testing.

        Tested it on a secure cluster, and the appropriate log file is created.

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - Updating the patch after testing. Tested it on a secure cluster, and the appropriate log file is created.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12536148/HADOOP-8552_branch1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1194//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12536148/HADOOP-8552_branch1.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1194//console This message is automatically generated.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        +1

        Show
        tucu00 Alejandro Abdelnur added a comment - +1
        Hide
        devaraj Devaraj Das added a comment -

        Hi Karthik, is this on the client or on the server side? (Guessing its on client.. please confirm). In general, the audit log stuff doesn't make sense on the client side. It's meant to be used on the server side only (and in deployments I know about, the security audit logging is turned off on the client side).
        Your patch will work though. But I'll note that it might be introducing compatibility issues due to the filename change of the log file (if someone is collecting logs based on file names, etc.).

        Show
        devaraj Devaraj Das added a comment - Hi Karthik, is this on the client or on the server side? (Guessing its on client.. please confirm). In general, the audit log stuff doesn't make sense on the client side. It's meant to be used on the server side only (and in deployments I know about, the security audit logging is turned off on the client side). Your patch will work though. But I'll note that it might be introducing compatibility issues due to the filename change of the log file (if someone is collecting logs based on file names, etc.).
        Hide
        kkambatl Karthik Kambatla (Inactive) added a comment -

        Devaraj, thanks for the feedback.

        It is both on the client/server side. By server side, I mean for the jobtracker/namenode. Thanks for pointing the potential compatibility issue, I agree we need to note the incompatibility in log file change.

        Show
        kkambatl Karthik Kambatla (Inactive) added a comment - Devaraj, thanks for the feedback. It is both on the client/server side. By server side, I mean for the jobtracker/namenode. Thanks for pointing the potential compatibility issue, I agree we need to note the incompatibility in log file change.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        Devaraj, are you OK with this patch after Karthik's clarifications?

        Show
        tucu00 Alejandro Abdelnur added a comment - Devaraj, are you OK with this patch after Karthik's clarifications?
        Hide
        devaraj Devaraj Das added a comment -

        Yes.

        Show
        devaraj Devaraj Das added a comment - Yes.
        Hide
        tucu00 Alejandro Abdelnur added a comment -

        Thanks Karthik. Committed to trunk, branch-1 and branch-2.

        Show
        tucu00 Alejandro Abdelnur added a comment - Thanks Karthik. Committed to trunk, branch-1 and branch-2.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2545/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2545/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2480 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2480/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = SUCCESS
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2480 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2480/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2500 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2500/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2500 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2500/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1106/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1106 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1106/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1139 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1139/)
        HADOOP-8552. Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151)

        Result = FAILURE
        tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1139 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1139/ ) HADOOP-8552 . Conflict: Same security.log.file for multiple users. (kkambatl via tucu) (Revision 1362151) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362151 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/packages/templates/conf/log4j.properties
        Hide
        sureshms Suresh Srinivas added a comment -

        Alejandro, when committing incompatible changes, could you please add the change description in CHANGES.txt under INCOMPATIBLE CHANGES section. Also could you please add release notes on what is incompatible here and how to get around it.

        Show
        sureshms Suresh Srinivas added a comment - Alejandro, when committing incompatible changes, could you please add the change description in CHANGES.txt under INCOMPATIBLE CHANGES section. Also could you please add release notes on what is incompatible here and how to get around it.
        Hide
        sureshms Suresh Srinivas added a comment -

        I also added this change in CHANGES.txt in branch 1.1.

        Show
        sureshms Suresh Srinivas added a comment - I also added this change in CHANGES.txt in branch 1.1.
        Hide
        mattf Matt Foley added a comment -

        Closed upon release of Hadoop-1.1.0.

        Show
        mattf Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

          People

          • Assignee:
            kasha Karthik Kambatla
            Reporter:
            kasha Karthik Kambatla
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development