Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2006

ability to support storing extended attributes per file

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HDFS XAttrs (HDFS-2006)
    • Fix Version/s: 2.5.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      It would be nice if HDFS provides a feature to store extended attributes for files, similar to the one described here: http://en.wikipedia.org/wiki/Extended_file_attributes.
      The challenge is that it has to be done in such a way that a site not using this feature does not waste precious memory resources in the namenode.

        Attachments

        1. ExtendedAttributes.html
          22 kB
          Uma Maheswara Rao G
        2. HDFS-2006-Branch-2-Merge.patch
          278 kB
          Uma Maheswara Rao G
        3. HDFS-2006-Merge-1.patch
          258 kB
          Uma Maheswara Rao G
        4. HDFS-2006-Merge-2.patch
          266 kB
          Uma Maheswara Rao G
        5. HDFS-XAttrs-Design-1.pdf
          402 kB
          Yi Liu
        6. HDFS-XAttrs-Design-2.pdf
          424 kB
          Yi Liu
        7. HDFS-XAttrs-Design-3.pdf
          434 kB
          Yi Liu
        8. Test-Plan-for-Extended-Attributes-1.pdf
          305 kB
          Yi Liu
        9. xattrs.1.patch
          138 kB
          Yi Liu
        10. xattrs.patch
          129 kB
          Yi Liu

          Issue Links

          1.
          Protobuf for XAttr and client-side implementation Sub-task Resolved Yi Liu
          2.
          Implement XAttr as a INode feature. Sub-task Resolved Yi Liu
          3.
          Namenode server-side storage for XAttrs Sub-task Resolved Yi Liu
          4.
          HDFS implementation of FileContext API for XAttrs. Sub-task Resolved Yi Liu
          5.
          Shift XAttr helper code out for reuse. Sub-task Resolved Yi Liu
          6.
          NameNode: persist XAttrs in fsimage and record XAttrs modifications to edit log. Sub-task Resolved Yi Liu
          7.
          XML based End-to-End test for getfattr and setfattr commands Sub-task Resolved Yi Liu
          8.
          Test cases for XAttrs Sub-task Resolved Yi Liu
          9.
          Javadocs for Xattrs apis in DFSClient and other minor fixups Sub-task Resolved Charles Lamb
          10.
          ClientProtocol#setXattr should not be annotated idempotent Sub-task Resolved Uma Maheswara Rao G
          11.
          TestOfflineEditsViewer for XAttr Sub-task Resolved Yi Liu
          12.
          Maximum limit on the size of an xattr Sub-task Resolved Yi Liu
          13.
          Optimize OP_SET_XATTRS by persisting single Xattr entry per setXattr/removeXattr api call Sub-task Resolved Yi Liu
          14.
          fix TestNamenodeRetryCache and TestRetryCacheWithHA failures Sub-task Resolved Uma Maheswara Rao G
          15.
          SetXattr should persist rpcIDs for handling retrycache with Namenode restart and HA Sub-task Resolved Uma Maheswara Rao G
          16.
          FsImage loading failed with RemoveXattr op Sub-task Resolved Uma Maheswara Rao G
          17.
          Clean up audit logging in FSNamesystem xattr methods Sub-task Resolved Andrew Wang
          18.
          Write end user documentation for xattrs. Sub-task Resolved Andrew Wang
          19.
          Handle setXattr rpcIDs for OfflineEditsViewer Sub-task Resolved Uma Maheswara Rao G
          20.
          Remove support for extended attributes on symlinks Sub-task Resolved Charles Lamb
          21.
          Unify xattr name and value limits into a single limit Sub-task Resolved Andrew Wang
          22.
          Support extended attributes via WebHDFS Sub-task Resolved Yi Liu
          23.
          setXAttr should require the user to be the owner of the file or directory Sub-task Resolved Charles Lamb
          24.
          Should have different size limits for different XAttr namespaces (user, trusted, security and system). Sub-task Resolved Yi Liu
          25.
          xattr names erroneously handled as case-insensitive. Sub-task Resolved Charles Lamb
          26.
          xattr modification operations are based on state of latest snapshot instead of current version of inode. Sub-task Resolved Andrew Wang
          27.
          DFSClient unwraps AclException in xattr methods, but those methods cannot throw AclException. Sub-task Resolved Andrew Wang
          28.
          Interface audience and stability annotations missing from several new classes related to xattrs. Sub-task Resolved Andrew Wang

            Activity

              People

              • Assignee:
                hitliuyi Yi Liu
                Reporter:
                dhruba dhruba borthakur
              • Votes:
                0 Vote for this issue
                Watchers:
                47 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: