[HDFS-4750] Support NFSv3 interface to HDFS

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Component/s: nfs
• Assignee: Brandon Li
• Reporter: Brandon Li
• Created: 25/Apr/13 00:44
• Updated: 11/Jun/16 08:56
• Resolved: 18/Nov/13 17:27

Description

Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration with client’s file system makes it difficult for users and impossible for some applications to access HDFS. NFS interface support is one way for HDFS to have such easy integration.

This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client, webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications and use cases.

We will upload the design document and the initial implementation.

Issue Links

• duplicates: HDFS-252 Export the HDFS file system through a NFS protocol
• is related to: HDFS-5804 HDFS NFS Gateway fails to mount and proxy when using Kerberos
• is related to: HADOOP-9806 PortmapInterface should check if the procedure is out-of-range
• is related to: HDFS-9164 hdfs-nfs connector fails on O_TRUNC
• is related to: HDFS-5523 Support multiple subdirectory exports in HDFS NFS gateway
• is related to: HDFS-5539 NFS gateway secuirty enhancement
• is related to: HADOOP-9509 Implement ONCRPC and XDR
• relates to: HADOOP-9515 Add general interface for NFS and Mount

Sub-Tasks

• Provide HDFS based NFSv3 and Mountd implementation: Closed
• Add script changes/utility for starting NFS gateway: Closed
• Add NFS server export table to control export by hostname or IP range: Closed
• Use enum for nfs constants: Closed
• Move IO operations out of locking in OpenFileCtx: Closed
• Support symlink operations: Closed
• Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment: Closed
• Change hdfs-nfs parent project to hadoop-project: Closed
• Support file append in NFSv3 gateway to enable data streaming to HDFS: Closed
• Add namespace ID and snapshot ID into fileHandle to support Federation and Snapshot: Open
• Refactor o.a.h.nfs to support different types of authentications: Closed
• Support dotdot name in NFS LOOKUP operation: Closed
• Fix array copy error in Readdir and Readdirplus responses: Closed
• Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error: Closed
• MNT EXPORT should give the full group list which can mount the exports: Closed
• Improve WriteManager for processing stable write requests and commit requests: Resolved
• NFS should create input stream for a file and try to share it with multiple read requests: Closed
• Handle race condition for writes: Resolved
• Add more debug trace for NFS READ and WRITE: Closed
• Include NFS jars in the maven assembly: Resolved
• Refactor RpcMessage and NFS3Response to support different types of authentication information: Closed
• Introduce RpcInfo to decouple XDR classes from the RPC API: Closed
• Move RpcFrameDecoder out of the public API: Closed
• Remove excessive copying due to XDR: Resolved
• Fix dumper thread which may die silently: Closed
• Stable write is not handled correctly in someplace: Closed
• Support client which combines appended data with old data before sends it to NFS server: Closed
• COMMIT request should not block: Closed
• Make Hadoop nfs server port and mount daemon port configurable: Closed
• Close idle connections in portmap: Closed
• fix readdir and readdirplus for large directories: Closed
• should do hsync for a commit request even there is no pending writes: Closed
• add HDFS NFS user guide: Closed
• Add OpenFileCtx cache: Closed
• Add more unit tests for the inputstream cache: Open
• Add more unit tests for the data streaming: Open
• COMMIT handler should update the commit status after sync: Closed