Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-beta1
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      FileStatus and FsPermission Writable serialization is deprecated and its implementation (incompatibly) replaced with protocol buffers. The FsPermissionProto record moved from hdfs.proto to acl.proto. HdfsFileStatus is now a subtype of FileStatus. FsPermissionExtension with its associated flags for ACLs, encryption, and erasure coding has been deprecated; users should query these attributes on the FileStatus object directly. The FsPermission instance in AclStatus no longer retains or reports these extended attributes (likely unused).
      Show
      FileStatus and FsPermission Writable serialization is deprecated and its implementation (incompatibly) replaced with protocol buffers. The FsPermissionProto record moved from hdfs.proto to acl.proto. HdfsFileStatus is now a subtype of FileStatus. FsPermissionExtension with its associated flags for ACLs, encryption, and erasure coding has been deprecated; users should query these attributes on the FileStatus object directly. The FsPermission instance in AclStatus no longer retains or reports these extended attributes (likely unused).

      Description

      FileStatus was a Writable in Hadoop 2 and earlier. Originally, we used this to serialize it and send it over the wire. But in Hadoop 2 and later, we have the protobuf HdfsFileStatusProto which serves to serialize this information. The protobuf form is preferable, since it allows us to add new fields in a backwards-compatible way. Another issue is that already a lot of subclasses of FileStatus don't override the Writable methods of the superclass, breaking the interface contract that read(status.write) should be equal to the original status.

      In Hadoop 3, we should just make FileStatus serialize itself via protobuf so that we don't have to deal with these issues. It's probably too late to do this in Hadoop 2, since user code may be relying on the existing FileStatus serialization there.

        Attachments

        1. HDFS-6984.nowritable.patch
          8 kB
          Andrew Wang
        2. HDFS-6984.015.patch
          104 kB
          Chris Douglas
        3. HDFS-6984.014.patch
          103 kB
          Chris Douglas
        4. HDFS-6984.013.patch
          103 kB
          Chris Douglas
        5. HDFS-6984.012.patch
          102 kB
          Chris Douglas
        6. HDFS-6984.011.patch
          97 kB
          Chris Douglas
        7. HDFS-6984.010.patch
          97 kB
          Chris Douglas
        8. HDFS-6984.009.patch
          93 kB
          Chris Douglas
        9. HDFS-6984.008.patch
          87 kB
          Chris Douglas
        10. HDFS-6984.007.patch
          83 kB
          Chris Douglas
        11. HDFS-6984.006.patch
          83 kB
          Chris Douglas
        12. HDFS-6984.005.patch
          24 kB
          Chris Douglas
        13. HDFS-6984.004.patch
          21 kB
          Chris Douglas
        14. HDFS-6984.003.patch
          20 kB
          Chris Douglas
        15. HDFS-6984.002.patch
          7 kB
          Colin P. McCabe
        16. HDFS-6984.001.patch
          2 kB
          Colin P. McCabe

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                cmccabe Colin P. McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: