Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3177

Allow DFSClient to find out and use the CRC type being used for a file.

    Details

    • Hadoop Flags:
      Reviewed

      Description

      To support HADOOP-8060, DFSClient should be able to find out the checksum type being used for files in hdfs.
      In my prototype, DataTransferProtocol was extended to include the checksum type in the blockChecksum() response. DFSClient uses it in getFileChecksum() to determin the checksum type. Also append() can be configured to use the existing checksum type instead of the configured one.

      1. hdfs-3177.patch
        16 kB
        Kihwal Lee
      2. hdfs-3177-after-hadoop-8239.patch.txt
        29 kB
        Kihwal Lee
      3. hdfs-3177-after-hadoop-8239-8240.patch.txt
        27 kB
        Kihwal Lee
      4. hdfs-3177-branch2-trunk.patch.txt
        31 kB
        Kihwal Lee
      5. hdfs-3177-branch2-trunk.patch.txt
        29 kB
        Kihwal Lee
      6. hdfs-3177-branch2-trunk.patch.txt
        28 kB
        Kihwal Lee
      7. hdfs-3177-branch2-trunk.patch.txt
        26 kB
        Kihwal Lee
      8. hdfs-3177-branch2-trunk.patch.txt
        23 kB
        Kihwal Lee
      9. hdfs-3177-branch2-trunk.patch.txt
        23 kB
        Kihwal Lee
      10. hdfs-3177-branch2-trunk.patch.txt
        27 kB
        Kihwal Lee
      11. hdfs-3177-minimum-without-append-concat-branch-0.23.patch.txt
        7 kB
        Kihwal Lee
      12. hdfs-3177-minimum-without-append-concat-trunk-branch2.patch.txt
        6 kB
        Kihwal Lee
      13. hdfs-3177-with-hadoop-8239.patch.txt
        38 kB
        Kihwal Lee
      14. hdfs-3177-with-hadoop-8239-8240.patch.txt
        89 kB
        Kihwal Lee
      15. hdfs-3177-with-hadoop-8239-8240.patch.txt
        90 kB
        Kihwal Lee
      16. hdfs-3177-with-hadoop-8239-8240.patch.txt
        85 kB
        Kihwal Lee

        Issue Links

          Activity

          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 3.0.0 [ 12320356 ]
          Daryn Sharp made changes -
          Fix Version/s 0.23.3 [ 12320052 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.2.0-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Resolution Fixed [ 1 ]
          Kihwal Lee made changes -
          Kihwal Lee made changes -
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542214 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542203 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542182 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542155 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542113 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12542065 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-branch2-trunk.patch.txt [ 12541744 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-with-hadoop-8239.patch.txt [ 12541654 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177-after-hadoop-8239.patch.txt [ 12541652 ]
          Kihwal Lee made changes -
          Kihwal Lee made changes -
          Kihwal Lee made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Kihwal Lee made changes -
          Kihwal Lee made changes -
          Arun C Murthy made changes -
          Fix Version/s 0.23.3 [ 12320052 ]
          Fix Version/s 2.0.0-alpha [ 12320353 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Kihwal Lee made changes -
          Link This issue is blocked by HADOOP-8239 [ HADOOP-8239 ]
          Kihwal Lee made changes -
          Link This issue blocks HADOOP-8239 [ HADOOP-8239 ]
          Kihwal Lee made changes -
          Attachment hdfs-3177.patch [ 12521025 ]
          Kihwal Lee made changes -
          Link This issue is blocked by HADOOP-8240 [ HADOOP-8240 ]
          Kihwal Lee made changes -
          Link This issue blocks HADOOP-8239 [ HADOOP-8239 ]
          Kihwal Lee made changes -
          Field Original Value New Value
          Link This issue is part of HADOOP-8060 [ HADOOP-8060 ]
          Kihwal Lee created issue -

            People

            • Assignee:
              Kihwal Lee
              Reporter:
              Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development