Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-5683

Better audit log messages for caching operations

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.3.0
    • Fix Version/s: 2.5.0
    • Component/s: namenode
    • Labels:

      Description

      Right now the caching audit logs aren't that useful, e.g.

      2013-12-18 14:14:54,423 INFO  FSNamesystem.audit (FSNamesystem.java:logAuditMessage(7362)) - allowed=true	ugi=andrew (auth:SIMPLE)	ip=/127.0.0.1	cmd=addCacheDirective	src=null	dst=null	perm=null
      

      It'd be good to include some more information when possible, like the path, pool, id, etc.

        Activity

        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1780 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1780/)
        HDFS-5683. Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1780 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1780/ ) HDFS-5683 . Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #1754 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1754/)
        HDFS-5683. Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1754 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1754/ ) HDFS-5683 . Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #562 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/562/)
        HDFS-5683. Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #562 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/562/ ) HDFS-5683 . Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #5606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5606/)
        HDFS-5683. Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054)

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5606/ ) HDFS-5683 . Better audit log messages for caching operations. Contributed by Abhiraj Butala. (wang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1595054 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        Abhiraj Butala added a comment -

        No problem Andrew, thank you for reviewing!

        Show
        Abhiraj Butala added a comment - No problem Andrew, thank you for reviewing!
        Hide
        Andrew Wang added a comment -

        Committed to trunk and branch-2, thanks for the contribution Abhiraj.

        Show
        Andrew Wang added a comment - Committed to trunk and branch-2, thanks for the contribution Abhiraj.
        Hide
        Andrew Wang added a comment -

        Sorry for letting this sit so long Abhiraj, this patch looks good to me. Audit logging is very file-centric right now so the semantics don't fit quite right with the caching methods, but this looks like a strict improvement over what we have right now.

        +1 will commit shortly.

        Show
        Andrew Wang added a comment - Sorry for letting this sit so long Abhiraj, this patch looks good to me. Audit logging is very file-centric right now so the semantics don't fit quite right with the caching methods, but this looks like a strict improvement over what we have right now. +1 will commit shortly.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12637680/HDFS-5683.001.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6556//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6556//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12637680/HDFS-5683.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6556//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6556//console This message is automatically generated.
        Hide
        Abhiraj Butala added a comment -

        I addressed the above issue and made appropriate changes to FSNamesystem.java. Giving below some examples of the caching audit logs with the patch:

        14/03/29 02:00:59 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=addCacheDirective	src={id: 8, path: /user/abutala/abhiraj, replication: 1, pool: pool7, expiration: 73071270-05-24T21:49:13-0700}	dst=null	perm=null
        14/03/29 02:01:49 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=modifyCacheDirective	src={id: 8}	dst={id: 8, path: /user/abutala/abhiraj/tmp2}	perm=null
        14/03/29 02:03:35 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=listCacheDirectives	src={}	dst=null	perm=null
        14/03/29 02:03:47 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=listCacheDirectives	src={pool: pool2}	dst=null	perm=null
        14/03/29 02:04:02 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=listCacheDirectives	src={path: /user/abutala/abhiraj, pool: pool2}	dst=null	perm=null
        14/03/29 02:05:54 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=removeCacheDirective	src={id: 8}	dst=null	perm=null
        14/03/29 02:08:44 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=addCachePool	src={poolName:pool10, ownerName:abutala, groupName:abutala, mode:0755, limit:9223372036854775807, maxRelativeExpiryMs:2305843009213693951}	dst=null	perm=null
        14/03/29 02:09:58 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=modifyCachePool	src={poolName: pool10}	dst={poolName:pool10, ownerName:null, groupName:null, mode:0666, limit:null, maxRelativeExpiryMs:null}	perm=null
        14/03/29 02:11:21 INFO FSNamesystem.audit: allowed=true	ugi=abutala (auth:SIMPLE)	ip=/127.0.0.1	cmd=removeCachePool	src={poolName: pool10}	dst=null	perm=null
        

        For modifyCacheDirective and modifyCachePool, I put the final changes in 'dst' section and the 'src' section only has the id or pool name being modified respectively. Also, not including any tests as this is just an update to the logs.

        Kindly review and let me know if there are any issues. Thank you!

        Show
        Abhiraj Butala added a comment - I addressed the above issue and made appropriate changes to FSNamesystem.java. Giving below some examples of the caching audit logs with the patch: 14/03/29 02:00:59 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=addCacheDirective src={id: 8, path: /user/abutala/abhiraj, replication: 1, pool: pool7, expiration: 73071270-05-24T21:49:13-0700} dst=null perm=null 14/03/29 02:01:49 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=modifyCacheDirective src={id: 8} dst={id: 8, path: /user/abutala/abhiraj/tmp2} perm=null 14/03/29 02:03:35 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=listCacheDirectives src={} dst=null perm=null 14/03/29 02:03:47 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=listCacheDirectives src={pool: pool2} dst=null perm=null 14/03/29 02:04:02 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=listCacheDirectives src={path: /user/abutala/abhiraj, pool: pool2} dst=null perm=null 14/03/29 02:05:54 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=removeCacheDirective src={id: 8} dst=null perm=null 14/03/29 02:08:44 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=addCachePool src={poolName:pool10, ownerName:abutala, groupName:abutala, mode:0755, limit:9223372036854775807, maxRelativeExpiryMs:2305843009213693951} dst=null perm=null 14/03/29 02:09:58 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=modifyCachePool src={poolName: pool10} dst={poolName:pool10, ownerName:null, groupName:null, mode:0666, limit:null, maxRelativeExpiryMs:null} perm=null 14/03/29 02:11:21 INFO FSNamesystem.audit: allowed=true ugi=abutala (auth:SIMPLE) ip=/127.0.0.1 cmd=removeCachePool src={poolName: pool10} dst=null perm=null For modifyCacheDirective and modifyCachePool, I put the final changes in 'dst' section and the 'src' section only has the id or pool name being modified respectively. Also, not including any tests as this is just an update to the logs. Kindly review and let me know if there are any issues. Thank you!
        Hide
        Abhiraj Butala added a comment -

        Hey Andrew,

        I am hoping to provide a fix for this issue and I would really appreciate if you can help me with a few beginner questions I have:

        I tried following steps to observe the caching audit log messages in hdfs-audit.log, but I don't see the logs being generated:
        a) Compiled and installed the latest hadoop-trunk.
        b) Updated the core-site.xml as per the documentation.
        c) Updated the $HADOOP_CONF_DIR/log4j.properties to direct hdfs audit logs to RFAAUDIT
        d) Started namenode and datanode. I could see the hdfs-audit.log file being generated in the $HADOOP_LOG_DIR/ as expected.
        e) Added a directory and a file in hdfs using 'hdfs dfs' commands.
        f) Created a cache pool: 'hdfs cacheadmin -addPool pool1'
        g) Added a cache directive: 'hdfs cacheadmin -addDirective -path [path added above] -pool pool1'

        I was hoping steps e), f) and g) would log the audit messages in hdfs-audit.log, but I did not see any logs there. Am I missing anything? Or could it be that my audit logging is not setup correctly?

        Thank you for your help!

        Show
        Abhiraj Butala added a comment - Hey Andrew, I am hoping to provide a fix for this issue and I would really appreciate if you can help me with a few beginner questions I have: I tried following steps to observe the caching audit log messages in hdfs-audit.log, but I don't see the logs being generated: a) Compiled and installed the latest hadoop-trunk. b) Updated the core-site.xml as per the documentation. c) Updated the $HADOOP_CONF_DIR/log4j.properties to direct hdfs audit logs to RFAAUDIT d) Started namenode and datanode. I could see the hdfs-audit.log file being generated in the $HADOOP_LOG_DIR/ as expected. e) Added a directory and a file in hdfs using 'hdfs dfs' commands. f) Created a cache pool: 'hdfs cacheadmin -addPool pool1' g) Added a cache directive: 'hdfs cacheadmin -addDirective -path [path added above] -pool pool1' I was hoping steps e), f) and g) would log the audit messages in hdfs-audit.log, but I did not see any logs there. Am I missing anything? Or could it be that my audit logging is not setup correctly? Thank you for your help!

          People

          • Assignee:
            Abhiraj Butala
            Reporter:
            Andrew Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development