Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12981

Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and core-default.xml

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.0.0-alpha1
    • None
    • documentation, tools

    Description

      It seems all properties defined in S3NativeFileSystemConfigKeys are not used. Those properties are prefixed by s3native, and the current s3native properties are all prefixed by fs.s3n, so this is likely not used currently. Additionally, core-default.xml has the description of these unused properties:

      <!-- s3native File System -->
      
      <property>
        <name>s3native.stream-buffer-size</name>
        <value>4096</value>
        <description>The size of buffer to stream files.
        The size of this buffer should probably be a multiple of hardware
        page size (4096 on Intel x86), and it determines how much data is
        buffered during read and write operations.</description>
      </property>
      
      <property>
        <name>s3native.bytes-per-checksum</name>
        <value>512</value>
        <description>The number of bytes per checksum.  Must not be larger than
        s3native.stream-buffer-size</description>
      </property>
      
      <property>
        <name>s3native.client-write-packet-size</name>
        <value>65536</value>
        <description>Packet size for clients to write</description>
      </property>
      
      <property>
        <name>s3native.blocksize</name>
        <value>67108864</value>
        <description>Block size</description>
      </property>
      
      <property>
        <name>s3native.replication</name>
        <value>3</value>
        <description>Replication factor</description>
      </property>
      

      I think they should be removed (or deprecated) to avoid confusion if these properties are defunct.

      Attachments

        1. HADOOP-12981.002.patch
          5 kB
          Wei-Chiu Chuang
        2. HADOOP-12981.001.patch
          4 kB
          Wei-Chiu Chuang

        Issue Links

          Activity

            People

              weichiu Wei-Chiu Chuang
              weichiu Wei-Chiu Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: