Details

      Description

      Webhdfs returns malformed json for directories that exceed the conf dfs.ls.limit value. The streaming object returned by NamenodeWebhdfsMethods#getListingStream will repeatedly call getListing for each segment of the directory listing. getListingStream runs within the remote user's ugi and acquires the first segment of the directory, then returns a streaming object. The streaming object is later executed outside of the user's ugi. Luckily it runs as the host service principal (ie. host/namenode@REALM) so the result is permission denied for the "host" user:

      org.apache.hadoop.security.AccessControlException: Permission denied: user=host, access=EXECUTE, inode="/path":someuser:group:drwx------
      

      The exception causes the streamer to prematurely abort the json output leaving it malformed. Meanwhile, the client sees the cryptic:

      java.lang.IllegalStateException: unexpected end of array
              at org.mortbay.util.ajax.JSON.parseArray(JSON.java:902)
              [...]
              at
      org.apache.hadoop.hdfs.web.WebHdfsFileSystem.jsonParse(WebHdfsFileSystem.java:242)
              at
      org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:441)
              at
      org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:717)
              [...]
      
      1. HDFS-4649.branch-23.patch
        6 kB
        Daryn Sharp
      2. HDFS-4649.branch-23.patch
        3 kB
        Daryn Sharp
      3. HDFS-4649.patch
        6 kB
        Daryn Sharp

        Activity

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development