[HADOOP-12981] Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and core-default.xml

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.0.0-alpha1
• Component/s: documentation, tools
• Labels: aws
• Assignee: Wei-Chiu Chuang
• Reporter: Wei-Chiu Chuang
• Created: 30/Mar/16 20:38
• Updated: 21/Sep/17 13:55
• Resolved: 21/Sep/17 13:55

Description

It seems all properties defined in S3NativeFileSystemConfigKeys are not used. Those properties are prefixed by s3native, and the current s3native properties are all prefixed by fs.s3n, so this is likely not used currently. Additionally, core-default.xml has the description of these unused properties:

<!-- s3native File System -->

<property>
  <name>s3native.stream-buffer-size</name>
  <value>4096</value>
  <description>The size of buffer to stream files.
  The size of this buffer should probably be a multiple of hardware
  page size (4096 on Intel x86), and it determines how much data is
  buffered during read and write operations.</description>
</property>

<property>
  <name>s3native.bytes-per-checksum</name>
  <value>512</value>
  <description>The number of bytes per checksum.  Must not be larger than
  s3native.stream-buffer-size</description>
</property>

<property>
  <name>s3native.client-write-packet-size</name>
  <value>65536</value>
  <description>Packet size for clients to write</description>
</property>

<property>
  <name>s3native.blocksize</name>
  <value>67108864</value>
  <description>Block size</description>
</property>

<property>
  <name>s3native.replication</name>
  <value>3</value>
  <description>Replication factor</description>
</property>

I think they should be removed (or deprecated) to avoid confusion if these properties are defunct.

Issue Links

• is depended upon by: HADOOP-13204 Über-jira: S3a phase III: scale and tuning
• is superceded by: HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs