Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2006

ability to support storing extended attributes per file

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • HDFS XAttrs (HDFS-2006)
    • 2.5.0
    • namenode
    • None
    • Reviewed

    Description

      It would be nice if HDFS provides a feature to store extended attributes for files, similar to the one described here: http://en.wikipedia.org/wiki/Extended_file_attributes.
      The challenge is that it has to be done in such a way that a site not using this feature does not waste precious memory resources in the namenode.

      Attachments

        1. ExtendedAttributes.html
          22 kB
          Uma Maheswara Rao G
        2. HDFS-2006-Branch-2-Merge.patch
          278 kB
          Uma Maheswara Rao G
        3. HDFS-2006-Merge-1.patch
          258 kB
          Uma Maheswara Rao G
        4. HDFS-2006-Merge-2.patch
          266 kB
          Uma Maheswara Rao G
        5. HDFS-XAttrs-Design-1.pdf
          402 kB
          Yi Liu
        6. HDFS-XAttrs-Design-2.pdf
          424 kB
          Yi Liu
        7. HDFS-XAttrs-Design-3.pdf
          434 kB
          Yi Liu
        8. Test-Plan-for-Extended-Attributes-1.pdf
          305 kB
          Yi Liu
        9. xattrs.1.patch
          138 kB
          Yi Liu
        10. xattrs.patch
          129 kB
          Yi Liu

        Issue Links

        1.
        Protobuf for XAttr and client-side implementation Sub-task Resolved Yi Liu Actions
        2.
        Implement XAttr as a INode feature. Sub-task Resolved Yi Liu Actions
        3.
        Namenode server-side storage for XAttrs Sub-task Resolved Yi Liu Actions
        4.
        HDFS implementation of FileContext API for XAttrs. Sub-task Resolved Yi Liu Actions
        5.
        Shift XAttr helper code out for reuse. Sub-task Resolved Yi Liu Actions
        6.
        NameNode: persist XAttrs in fsimage and record XAttrs modifications to edit log. Sub-task Resolved Yi Liu Actions
        7.
        XML based End-to-End test for getfattr and setfattr commands Sub-task Resolved Yi Liu Actions
        8.
        Test cases for XAttrs Sub-task Resolved Yi Liu Actions
        9.
        Javadocs for Xattrs apis in DFSClient and other minor fixups Sub-task Resolved Charles Lamb Actions
        10.
        ClientProtocol#setXattr should not be annotated idempotent Sub-task Resolved Uma Maheswara Rao G Actions
        11.
        TestOfflineEditsViewer for XAttr Sub-task Resolved Yi Liu Actions
        12.
        Maximum limit on the size of an xattr Sub-task Resolved Yi Liu Actions
        13.
        Optimize OP_SET_XATTRS by persisting single Xattr entry per setXattr/removeXattr api call Sub-task Resolved Yi Liu Actions
        14.
        fix TestNamenodeRetryCache and TestRetryCacheWithHA failures Sub-task Resolved Uma Maheswara Rao G Actions
        15.
        SetXattr should persist rpcIDs for handling retrycache with Namenode restart and HA Sub-task Resolved Uma Maheswara Rao G Actions
        16.
        FsImage loading failed with RemoveXattr op Sub-task Resolved Uma Maheswara Rao G Actions
        17.
        Clean up audit logging in FSNamesystem xattr methods Sub-task Resolved Andrew Wang Actions
        18.
        Write end user documentation for xattrs. Sub-task Resolved Andrew Wang Actions
        19.
        Handle setXattr rpcIDs for OfflineEditsViewer Sub-task Resolved Uma Maheswara Rao G Actions
        20.
        Remove support for extended attributes on symlinks Sub-task Resolved Charles Lamb Actions
        21.
        Unify xattr name and value limits into a single limit Sub-task Resolved Andrew Wang Actions
        22.
        Support extended attributes via WebHDFS Sub-task Resolved Yi Liu Actions
        23.
        setXAttr should require the user to be the owner of the file or directory Sub-task Resolved Charles Lamb Actions
        24.
        Should have different size limits for different XAttr namespaces (user, trusted, security and system). Sub-task Resolved Yi Liu Actions
        25.
        xattr names erroneously handled as case-insensitive. Sub-task Resolved Charles Lamb Actions
        26.
        xattr modification operations are based on state of latest snapshot instead of current version of inode. Sub-task Resolved Andrew Wang Actions
        27.
        DFSClient unwraps AclException in xattr methods, but those methods cannot throw AclException. Sub-task Resolved Andrew Wang Actions
        28.
        Interface audience and stability annotations missing from several new classes related to xattrs. Sub-task Resolved Andrew Wang Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            hitliuyi Yi Liu
            dhruba Dhruba Borthakur
            Votes:
            0 Vote for this issue
            Watchers:
            47 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment