Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8240

Allow users to specify a checksum type on create()

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Per discussion in HADOOP-8060, a way for users to specify a checksum type on create() is needed. The way FileSystem cache works makes it impossible to use dfs.checksum.type to achieve this. Also checksum-related API is at Filesystem-level, so we prefer something at that level, not hdfs-specific one. Current proposal is to use CreatFlag.

        Attachments

        1. hadoop-8240.patch
          4 kB
          Kihwal Lee
        2. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        3. hadoop-8240-post-hadoop-8700-br2-trunk.patch.txt
          59 kB
          Kihwal Lee
        4. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        5. hadoop-8240-post-hadoop-8700-br2-trunk.patch.txt
          60 kB
          Kihwal Lee
        6. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        7. hadoop-8240-branch-0.23-alone.patch.txt
          50 kB
          Kihwal Lee

          Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3177 Allow DFSClient to find out and use the CRC type being used for a file.

          • Major - Major loss of function.
          • Closed
          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8700 Move the checksum type constants to an enum

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is part of

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-8060 Add a capability to discover and set checksum types per file.

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6542 WebHDFSFileSystem doesn't transmit desired checksum type

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-10237 Support specifying checksum type in WebHDFS/HTTPFS writers

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Patch Available
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9505 Specifying checksum type to NULL can cause write failures with AIOBE

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9114 After defined the dfs.checksum.type as the NULL, write file and hflush will through java.lang.ArrayIndexOutOfBoundsException

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

            Activity

              People

              • Assignee:
                kihwal Kihwal Lee
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: