Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: nfs
    • Labels:
      None

      Description

      Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration with client’s file system makes it difficult for users and impossible for some applications to access HDFS. NFS interface support is one way for HDFS to have such easy integration.

      This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client, webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications and use cases.

      We will upload the design document and the initial implementation.

      1. nfs-trunk.patch
        368 kB
        Brandon Li
      2. HDFS-4750.patch
        371 kB
        Brandon Li
      3. HADOOP-NFS-Proposal.pdf
        221 kB
        Brandon Li

        Issue Links

        1.
        Provide HDFS based NFSv3 and Mountd implementation Sub-task Closed Brandon Li
         
        2.
        Add script changes/utility for starting NFS gateway Sub-task Closed Brandon Li
         
        3.
        Add NFS server export table to control export by hostname or IP range Sub-task Closed Jing Zhao
         
        4.
        Use enum for nfs constants Sub-task Closed Tsz Wo Nicholas Sze
         
        5.
        Move IO operations out of locking in OpenFileCtx Sub-task Closed Jing Zhao
         
        6.
        Support symlink operations Sub-task Closed Brandon Li
         
        7.
        Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment Sub-task Closed Brandon Li
         
        8.
        Change hdfs-nfs parent project to hadoop-project Sub-task Closed Brandon Li
         
        9.
        Support file append in NFSv3 gateway to enable data streaming to HDFS Sub-task Closed Brandon Li
         
        10. Add namespace ID and snapshot ID into fileHandle to support Federation and Snapshot Sub-task Open Unassigned
         
        11.
        Refactor o.a.h.nfs to support different types of authentications Sub-task Closed Jing Zhao
         
        12.
        Support dotdot name in NFS LOOKUP operation Sub-task Closed Brandon Li
         
        13.
        Fix array copy error in Readdir and Readdirplus responses Sub-task Closed Brandon Li
         
        14.
        Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error Sub-task Closed Brandon Li
         
        15.
        MNT EXPORT should give the full group list which can mount the exports Sub-task Closed Brandon Li
         
        16.
        Improve WriteManager for processing stable write requests and commit requests Sub-task Resolved Jing Zhao
         
        17.
        NFS should create input stream for a file and try to share it with multiple read requests Sub-task Closed Haohui Mai
         
        18.
        Handle race condition for writes Sub-task Resolved Brandon Li
         
        19.
        Add more debug trace for NFS READ and WRITE Sub-task Closed Brandon Li
         
        20.
        Include NFS jars in the maven assembly Sub-task Resolved Unassigned
         
        21.
        Refactor RpcMessage and NFS3Response to support different types of authentication information Sub-task Closed Jing Zhao
         
        22.
        Introduce RpcInfo to decouple XDR classes from the RPC API Sub-task Resolved Haohui Mai
         
        23.
        Move RpcFrameDecoder out of the public API Sub-task Closed Haohui Mai
         
        24.
        Remove excessive copying due to XDR Sub-task Resolved Haohui Mai
         
        25.
        Fix dumper thread which may die silently Sub-task Closed Brandon Li
         
        26.
        Stable write is not handled correctly in someplace Sub-task Closed Brandon Li
         
        27.
        Support client which combines appended data with old data before sends it to NFS server Sub-task Resolved Brandon Li
         
        28.
        COMMIT request should not block Sub-task Closed Brandon Li
         
        29.
        Make Hadoop nfs server port and mount daemon port configurable Sub-task Resolved Jinghui Wang
         
        30.
        Close idle connections in portmap Sub-task Closed Haohui Mai
         
        31.
        fix readdir and readdirplus for large directories Sub-task Closed Brandon Li
         
        32.
        should do hsync for a commit request even there is no pending writes Sub-task Closed Brandon Li
         
        33.
        add HDFS NFS user guide Sub-task Closed Brandon Li
         
        34.
        Add OpenFileCtx cache Sub-task Closed Brandon Li
         
        35. Add more unit tests for the inputstream cache Sub-task Open Unassigned
         
        36. Add more unit tests for the data streaming Sub-task Open Unassigned
         
        37.
        COMMIT handler should update the commit status after sync Sub-task Closed Brandon Li
         

          Activity

          Brandon Li created issue -
          Brandon Li made changes -
          Field Original Value New Value
          Attachment HADOOP-NFS-Proposal.pdf [ 12580444 ]
          Brandon Li made changes -
          Link This issue duplicates HDFS-252 [ HDFS-252 ]
          Brandon Li made changes -
          Description Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration with client’s file system makes it difficult for users and impossible for some applications to access HDFS. NFS interface support is one way for HDFS to support such easy integration.

          This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client, webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications and use cases.

          We will upload the design document and the initial implementation.
          Access HDFS is usually done through HDFS Client or webHDFS. Lack of seamless integration with client’s file system makes it difficult for users and impossible for some applications to access HDFS. NFS interface support is one way for HDFS to have such easy integration.

          This JIRA is to track the NFS protocol support for accessing HDFS. With HDFS client, webHDFS and the NFS interface, HDFS will be easier to access and be able support more applications and use cases.

          We will upload the design document and the initial implementation.
          Brandon Li made changes -
          Link This issue is related to HADOOP-9509 [ HADOOP-9509 ]
          Brandon Li made changes -
          Link This issue relates to HADOOP-9515 [ HADOOP-9515 ]
          Brandon Li made changes -
          Attachment nfs-trunk.patch [ 12581119 ]
          Brandon Li made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Brandon Li made changes -
          Attachment HDFS-4750.patch [ 12581664 ]
          Tsz Wo Nicholas Sze made changes -
          Component/s nfs [ 12320727 ]
          Brandon Li made changes -
          Link This issue is related to HADOOP-9806 [ HADOOP-9806 ]
          Brandon Li made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Brandon Li made changes -
          Link This issue is related to HDFS-5523 [ HDFS-5523 ]
          Brandon Li made changes -
          Link This issue is related to HDFS-5539 [ HDFS-5539 ]
          Abin Shahab made changes -
          Link This issue is related to HDFS-5804 [ HDFS-5804 ]

            People

            • Assignee:
              Brandon Li
              Reporter:
              Brandon Li
            • Votes:
              0 Vote for this issue
              Watchers:
              43 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development