Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12981

Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and core-default.xml

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: documentation, tools
    • Labels:

      Description

      It seems all properties defined in S3NativeFileSystemConfigKeys are not used. Those properties are prefixed by s3native, and the current s3native properties are all prefixed by fs.s3n, so this is likely not used currently. Additionally, core-default.xml has the description of these unused properties:

      <!-- s3native File System -->
      
      <property>
        <name>s3native.stream-buffer-size</name>
        <value>4096</value>
        <description>The size of buffer to stream files.
        The size of this buffer should probably be a multiple of hardware
        page size (4096 on Intel x86), and it determines how much data is
        buffered during read and write operations.</description>
      </property>
      
      <property>
        <name>s3native.bytes-per-checksum</name>
        <value>512</value>
        <description>The number of bytes per checksum.  Must not be larger than
        s3native.stream-buffer-size</description>
      </property>
      
      <property>
        <name>s3native.client-write-packet-size</name>
        <value>65536</value>
        <description>Packet size for clients to write</description>
      </property>
      
      <property>
        <name>s3native.blocksize</name>
        <value>67108864</value>
        <description>Block size</description>
      </property>
      
      <property>
        <name>s3native.replication</name>
        <value>3</value>
        <description>Replication factor</description>
      </property>
      

      I think they should be removed (or deprecated) to avoid confusion if these properties are defunct.

        Attachments

        1. HADOOP-12981.001.patch
          4 kB
          Wei-Chiu Chuang
        2. HADOOP-12981.002.patch
          5 kB
          Wei-Chiu Chuang

          Issue Links

            Activity

              People

              • Assignee:
                weichiu Wei-Chiu Chuang
                Reporter:
                weichiu Wei-Chiu Chuang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: