Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6411

nfs-hdfs-gateway mount raises I/O error and hangs when a unauthorized user attempts to access it

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.4.1
    • Component/s: nfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We use the nfs-hdfs gateway to expose hdfs thru nfs.

      0) login as root, run nfs-hdfs gateway as a user, say, nfsserver.
      [root@zhongyi-test-cluster-desktop hdfs]# ls /hdfs
      backups hive mr-history system tmp user
      1) add a user nfs-test: adduser nfs-test(make sure that this user is not a proxyuser of nfsserver
      2) switch to test user: su - nfs-test
      3) access hdfs nfs gateway
      [nfs-test@zhongyi-test-cluster-desktop ~]$ ls /hdfs
      ls: cannot open directory /hdfs: Input/output error
      retry:
      [nfs-test@zhongyi-test-cluster-desktop ~]$ ls /hdfs
      ls: cannot access /hdfs: Stale NFS file handle
      4) switch back to root and access hdfs nfs gateway
      [nfs-test@zhongyi-test-cluster-desktop ~]$ exit
      logout
      [root@zhongyi-test-cluster-desktop hdfs]# ls /hdfs
      ls: cannot access /hdfs: Stale NFS file handle

      the nfsserver log indicates we hit an authorization error in the rpc handler; org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: nfsserver is not allowed to impersonate nfs-test
      and NFS3ERR_IO is returned, which explains why we see input/output error.
      One can catch the authorizationexception and return the correct error: NFS3ERR_ACCES to fix the error message on the client side but that doesn't seem to solve the mount hang issue though. When the mount hang happens, it stops printing nfsserver log which makes it more difficult to figure out the real cause of the hang. According to jstack and debugger, the nfsserver seems to be waiting for client requests

        Attachments

        1. HDFS-6411.patch
          1.0 kB
          Brandon Li
        2. HDFS-6411-branch-2.2.patch
          3 kB
          Zhongyi Xie
        3. tcpdump-HDFS-6411-Brandon.out
          0.5 kB
          Zhongyi Xie
        4. HDFS-6411.002.patch
          3 kB
          Brandon Li
        5. HDFS-6411.003.patch
          3 kB
          Brandon Li
        6. HDFS-6411.004.patch
          4 kB
          Brandon Li

          Issue Links

            Activity

              People

              • Assignee:
                brandonli Brandon Li
                Reporter:
                zhongyi-altiscale Zhongyi Xie
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: