Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8240

Allow users to specify a checksum type on create()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 0.23.3, 2.0.2-alpha
    • fs
    • None
    • Reviewed

    Description

      Per discussion in HADOOP-8060, a way for users to specify a checksum type on create() is needed. The way FileSystem cache works makes it impossible to use dfs.checksum.type to achieve this. Also checksum-related API is at Filesystem-level, so we prefer something at that level, not hdfs-specific one. Current proposal is to use CreatFlag.

      Attachments

        1. hadoop-8240.patch
          4 kB
          Kihwal Lee
        2. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        3. hadoop-8240-post-hadoop-8700-br2-trunk.patch.txt
          59 kB
          Kihwal Lee
        4. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        5. hadoop-8240-post-hadoop-8700-br2-trunk.patch.txt
          60 kB
          Kihwal Lee
        6. hadoop-8240-trunk-branch2.patch.txt
          58 kB
          Kihwal Lee
        7. hadoop-8240-branch-0.23-alone.patch.txt
          50 kB
          Kihwal Lee

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3177 Allow DFSClient to find out and use the CRC type being used for a file.

          • Major - Major loss of function.
          • Closed
          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-8700 Move the checksum type constants to an enum

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is part of

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-8060 Add a capability to discover and set checksum types per file.

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6542 WebHDFSFileSystem doesn't transmit desired checksum type

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HDFS-10237 Support specifying checksum type in WebHDFS/HTTPFS writers

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Patch Available
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9505 Specifying checksum type to NULL can cause write failures with AIOBE

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-9114 After defined the dfs.checksum.type as the NULL, write file and hflush will through java.lang.ArrayIndexOutOfBoundsException

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              kihwal Kihwal Lee
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: