Accumulo
  1. Accumulo
  2. ACCUMULO-1282

Monitor requires jumping through hadoop permissions hoops (and granting accumulo broad permissions)

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      The monitor's master status box requires getContentSummary(new Path("/")) on HDFS (otherwise see stack trace below). There doesn't seem to be any way to grant this permission to a particular user or change the permissions on /, so for this to work, you either need to run accumulo and hdfs as the same user or add accumulo's user to hadoop's supergroup. Either way, this seems like it's granting unnecessarily broad permissions to accumulo. Is there some other way to get the disk usage information out of hadoop with normal user-level permissions?

      Stack trace running as separate users without special permissions:

      2013-04-16 20:34:57,770 [servlets.BasicServlet] DEBUG:  org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, ac
      cess=READ_EXECUTE, inode="system":hadoop:supergroup:rwx------
      org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=READ_EXECUTE, inode="system":hadoop:supergroup:rwx-
      -----
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
              at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
              at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
              at org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:1438)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getContentSummary(DistributedFileSystem.java:251)
              at org.apache.accumulo.server.trace.TraceFileSystem.getContentSummary(TraceFileSystem.java:312)
              at org.apache.accumulo.server.monitor.servlets.DefaultServlet.doAccumuloTable(DefaultServlet.java:317)
              at org.apache.accumulo.server.monitor.servlets.DefaultServlet.pageBody(DefaultServlet.java:256)
              at org.apache.accumulo.server.monitor.servlets.BasicServlet.doGet(BasicServlet.java:61)
              at org.apache.accumulo.server.monitor.servlets.DefaultServlet.doGet(DefaultServlet.java:157)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
              at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
              at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
              at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
              at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
              at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
              at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
              at org.mortbay.jetty.Server.handle(Server.java:326)
              at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
              at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
              at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
              at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
              at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
              at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
              at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
      Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied: user=accumulo, access=READ_EXECUTE, inode="system":hadoop:supergroup:rwx--
      ----
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:199)
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkSubAccess(FSPermissionChecker.java:168)
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:137)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5468)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getContentSummary(FSNamesystem.java:2225)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.getContentSummary(NameNode.java:986)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:616)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:416)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
      
              at org.apache.hadoop.ipc.Client.call(Client.java:1107)
              at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
              at sun.proxy.$Proxy1.getContentSummary(Unknown Source)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:616)
              at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
              at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
              at sun.proxy.$Proxy1.getContentSummary(Unknown Source)
              at org.apache.hadoop.hdfs.DFSClient.getContentSummary(DFSClient.java:1436)
              ... 22 more
      

        Activity

        Hide
        Michael Berman added a comment -

        Hadoop 1.1.2, btw

        Show
        Michael Berman added a comment - Hadoop 1.1.2, btw
        Hide
        Keith Turner added a comment -

        Does this prevent the monitor page from displaying? If so maybe for 1.5 we can fix that, make it display something like N/A when it can not get the info.

        Show
        Keith Turner added a comment - Does this prevent the monitor page from displaying? If so maybe for 1.5 we can fix that, make it display something like N/A when it can not get the info.
        Hide
        John Vines added a comment -

        It does prevent the Accumulo Master window in the Overview page from displaying.

        Show
        John Vines added a comment - It does prevent the Accumulo Master window in the Overview page from displaying.
        Hide
        Eric Newton added a comment -

        If there's a problem fetching this info from hdfs, the table will just say "Unknown".

        Show
        Eric Newton added a comment - If there's a problem fetching this info from hdfs, the table will just say "Unknown".
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #82 (See https://builds.apache.org/job/Accumulo-1.5/82/)
        ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468995)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #82 (See https://builds.apache.org/job/Accumulo-1.5/82/ ) ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468995) Result = SUCCESS ecn : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5-Hadoop-2.0 #81 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/81/)
        ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468995)

        Result = FAILURE
        ecn :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5-Hadoop-2.0 #81 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/81/ ) ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468995) Result = FAILURE ecn : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #835 (See https://builds.apache.org/job/Accumulo-Trunk/835/)
        ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468997)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #835 (See https://builds.apache.org/job/Accumulo-Trunk/835/ ) ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468997) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #193 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/193/)
        ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468997)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #193 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/193/ ) ACCUMULO-1282 ignore permission problems getting disk usage information (Revision 1468997) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/monitor/servlets/DefaultServlet.java /accumulo/trunk/src
        Show
        Sean Busbey added a comment - start of thread discussing manually fixing permissions on 1.4.x and suggested workaround

          People

          • Assignee:
            Eric Newton
            Reporter:
            Michael Berman
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development