Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4970

Child tasks (try to) create security audit log files

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.1
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After HADOOP-8552, MR child tasks will attempt to create security audit log files with their user names. On an insecure cluster, this has no effect, but on a secure cluster, log4j will try to create log files for tasks with names like SecurityAuth-joeuser.log.

      1. MAPREDUCE-4970.patch
        5 kB
        Sandy Ryza
      2. MAPREDUCE-4970.patch
        5 kB
        Sandy Ryza
      3. MAPREDUCE-4970-1.patch
        6 kB
        Sandy Ryza

        Activity

        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop 1.2.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop 1.2.0.
        Hide
        Yesha Vora added a comment -

        opened new jira MAPREDUCE-5148

        Show
        Yesha Vora added a comment - opened new jira MAPREDUCE-5148
        Hide
        Yesha Vora added a comment -

        This patch is not compatible with older version. The main impact of this change is syslogs are missing on clusters which are configured with just log4j.properties. Also, stderr has the following WARN message:
        log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Child).
        log4j:WARN Please initialize the log4j system properly.

        Show
        Yesha Vora added a comment - This patch is not compatible with older version. The main impact of this change is syslogs are missing on clusters which are configured with just log4j.properties. Also, stderr has the following WARN message: log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.Child). log4j:WARN Please initialize the log4j system properly.
        Hide
        Alejandro Abdelnur added a comment - - edited

        Thanks Sandy. Committed to branch-1.

        Show
        Alejandro Abdelnur added a comment - - edited Thanks Sandy. Committed to branch-1.
        Hide
        Alejandro Abdelnur added a comment -

        +1

        Show
        Alejandro Abdelnur added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12568151/MAPREDUCE-4970-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3308//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12568151/MAPREDUCE-4970-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3308//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        Latest patch includes update to cluster_setup.xml

        Show
        Sandy Ryza added a comment - Latest patch includes update to cluster_setup.xml
        Hide
        Alejandro Abdelnur added a comment -

        Looks good, we need to update the docs mentioning this file and what is used for, in xdocs cluster_setup.xml.

        Show
        Alejandro Abdelnur added a comment - Looks good, we need to update the docs mentioning this file and what is used for, in xdocs cluster_setup.xml.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12568102/MAPREDUCE-4970.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3305//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12568102/MAPREDUCE-4970.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3305//console This message is automatically generated.
        Hide
        Sandy Ryza added a comment -

        This is difficult to write a test for, but I verified it on a pseudo-distributed cluster.

        Show
        Sandy Ryza added a comment - This is difficult to write a test for, but I verified it on a pseudo-distributed cluster.
        Hide
        Karthik Kambatla added a comment -

        +1 on the approach.

        Show
        Karthik Kambatla added a comment - +1 on the approach.
        Hide
        Sandy Ryza added a comment -

        I propose creating a task-log4j.properties without the SecurityAuth logger, and passing that instead of the default to the task. This is what MR2 does, with container-log4j.properties.

        Show
        Sandy Ryza added a comment - I propose creating a task-log4j.properties without the SecurityAuth logger, and passing that instead of the default to the task. This is what MR2 does, with container-log4j.properties.

          People

          • Assignee:
            Sandy Ryza
            Reporter:
            Sandy Ryza
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development