Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12981

Remove/deprecate s3native properties from S3NativeFileSystemConfigKeys and core-default.xml

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.0.0-alpha1
    • None
    • documentation, tools

    Description

      It seems all properties defined in S3NativeFileSystemConfigKeys are not used. Those properties are prefixed by s3native, and the current s3native properties are all prefixed by fs.s3n, so this is likely not used currently. Additionally, core-default.xml has the description of these unused properties:

      <!-- s3native File System -->
      
      <property>
        <name>s3native.stream-buffer-size</name>
        <value>4096</value>
        <description>The size of buffer to stream files.
        The size of this buffer should probably be a multiple of hardware
        page size (4096 on Intel x86), and it determines how much data is
        buffered during read and write operations.</description>
      </property>
      
      <property>
        <name>s3native.bytes-per-checksum</name>
        <value>512</value>
        <description>The number of bytes per checksum.  Must not be larger than
        s3native.stream-buffer-size</description>
      </property>
      
      <property>
        <name>s3native.client-write-packet-size</name>
        <value>65536</value>
        <description>Packet size for clients to write</description>
      </property>
      
      <property>
        <name>s3native.blocksize</name>
        <value>67108864</value>
        <description>Block size</description>
      </property>
      
      <property>
        <name>s3native.replication</name>
        <value>3</value>
        <description>Replication factor</description>
      </property>
      

      I think they should be removed (or deprecated) to avoid confusion if these properties are defunct.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            weichiu Wei-Chiu Chuang Assign to me
            weichiu Wei-Chiu Chuang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment