Hadoop Common
  1. Hadoop Common
  2. HADOOP-10610

Upgrade S3n fs.s3.buffer.dir to support multi directories

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.6.0
    • Component/s: fs/s3
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      fs.s3.buffer.dir defines the tmp folder where files will be written to before getting sent to S3. Right now this is limited to a single folder which causes to major issues.

      1. You need a drive with enough space to store all the tmp files at once
      2. You are limited to the IO speeds of a single drive

      This solution will resolve both and has been tested to increase the S3 write speed by 2.5x with 10 mappers on hs1.

      1. HADOOP_10610-2.patch
        2 kB
        Ted Malaska
      2. HADOOP-10610.patch
        2 kB
        Ted Malaska
      3. HDFS-6383.patch
        1 kB
        Ted Malaska

        Issue Links

          Activity

          Hide
          Ted Malaska added a comment -

          This solves the problem.

          Thanks to:
          Joe Prosser, Dave Wang, Andrei Savu
          and
          Govind Kamat for testing

          Show
          Ted Malaska added a comment - This solves the problem. Thanks to: Joe Prosser, Dave Wang, Andrei Savu and Govind Kamat for testing
          Hide
          Ted Malaska added a comment -

          Patch is ready

          Show
          Ted Malaska added a comment - Patch is ready
          Hide
          Aaron T. Myers added a comment -

          Thanks a lot for the patch, Ted. I've assigned it to you and will go see if I can figure out why the pre-commit Jenkins job hasn't run yet.

          Show
          Aaron T. Myers added a comment - Thanks a lot for the patch, Ted. I've assigned it to you and will go see if I can figure out why the pre-commit Jenkins job hasn't run yet.
          Hide
          Aaron T. Myers added a comment -

          Not sure why it didn't run, but I've just kicked Jenkins manually. Here's a link to the pre-commit build for this patch:

          https://builds.apache.org/job/PreCommit-HDFS-Build/6897/

          Show
          Aaron T. Myers added a comment - Not sure why it didn't run, but I've just kicked Jenkins manually. Here's a link to the pre-commit build for this patch: https://builds.apache.org/job/PreCommit-HDFS-Build/6897/
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12644622/HDFS-6383.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The following test timeouts occurred in hadoop-common-project/hadoop-common:

          org.apache.hadoop.http.TestHttpServer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6897//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6897//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12644622/HDFS-6383.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The following test timeouts occurred in hadoop-common-project/hadoop-common: org.apache.hadoop.http.TestHttpServer +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6897//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6897//console This message is automatically generated.
          Hide
          David S. Wang added a comment -

          Thanks Ted for the patch.

          We should probably use the LocalDirAllocator like what s3a uses. That seems to be the proper way to do this in Hadoop.

          Show
          David S. Wang added a comment - Thanks Ted for the patch. We should probably use the LocalDirAllocator like what s3a uses. That seems to be the proper way to do this in Hadoop.
          Hide
          Steve Loughran added a comment -

          moved from -HDFS to -COMMON, labelled as fs/s3

          Show
          Steve Loughran added a comment - moved from -HDFS to -COMMON, labelled as fs/s3
          Hide
          Steve Loughran added a comment -

          The overall concept makes sense

          1. LocalDirAllocator contains the logic to worry about file capacity, writeability &c. The more disks you list, the more likely a disk is to fail, the more you need that code. This patch currently just bails out if one dest dir isn't there, even if others may be present.
          2. This would be an ideal time to move "fs.s3.buffer.dir" from an inline string to a constant where it can be referred to in hadoop and external code

          Recommended tests

          1. a test with > 1 directory in the args
          2. basic handling of "erroneous" inputs. e.g trailing commas in options
          3. bad directories in input paths
          Show
          Steve Loughran added a comment - The overall concept makes sense LocalDirAllocator contains the logic to worry about file capacity, writeability &c. The more disks you list, the more likely a disk is to fail, the more you need that code. This patch currently just bails out if one dest dir isn't there, even if others may be present. This would be an ideal time to move "fs.s3.buffer.dir" from an inline string to a constant where it can be referred to in hadoop and external code Recommended tests a test with > 1 directory in the args basic handling of "erroneous" inputs. e.g trailing commas in options bad directories in input paths
          Hide
          Juan Yu added a comment -

          Just wondering why don't cache the tmp dir list? do we have to parse the config to get the list for every newBackupFile() call?

          Show
          Juan Yu added a comment - Just wondering why don't cache the tmp dir list? do we have to parse the config to get the list for every newBackupFile() call?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12644622/HDFS-6383.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3937//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3937//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12644622/HDFS-6383.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3937//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3937//console This message is automatically generated.
          Hide
          Ted Malaska added a comment -

          Sorry I have been slow on this. I will have the new patch sometime this weekend.

          Thanks again for the help.

          Show
          Ted Malaska added a comment - Sorry I have been slow on this. I will have the new patch sometime this weekend. Thanks again for the help.
          Hide
          Andrew Wang added a comment -

          Hey Ted,

          Note that we already have Configuration.getTrimmedStrings, no need to split yourself, and thus no need to write unit tests for splitting logic

          Show
          Andrew Wang added a comment - Hey Ted, Note that we already have Configuration.getTrimmedStrings, no need to split yourself, and thus no need to write unit tests for splitting logic
          Hide
          Ted Malaska added a comment -

          Here is the updated patch based on your feed back.

          Thank you again for allowing me to help

          Let me know if there are any additional changes needed.

          Show
          Ted Malaska added a comment - Here is the updated patch based on your feed back. Thank you again for allowing me to help Let me know if there are any additional changes needed.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12645414/HADOOP-10610.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.s3native.TestS3NInMemoryFileSystem

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3952//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3952//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12645414/HADOOP-10610.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.s3native.TestS3NInMemoryFileSystem +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3952//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3952//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Ted - could you please rev this patch so that it passes the two failed unit tests? They seem pretty relevant to the changes that this patch is making.

          Show
          Aaron T. Myers added a comment - Hey Ted - could you please rev this patch so that it passes the two failed unit tests? They seem pretty relevant to the changes that this patch is making.
          Hide
          Ted Malaska added a comment -

          Hey ATM,

          Yes I will review it this weekend. Thanks

          Show
          Ted Malaska added a comment - Hey ATM, Yes I will review it this weekend. Thanks
          Hide
          Ted Malaska added a comment -

          Fixed the null pointer exception related to lDirAlloc.

          Show
          Ted Malaska added a comment - Fixed the null pointer exception related to lDirAlloc.
          Hide
          Ted Malaska added a comment -

          Hey ATM,

          Sorry for missing that. I updated the code and submitted the new patch.

          Ted Malaska

          On Fri, Jun 13, 2014 at 8:11 PM, Aaron T. Myers (JIRA) <jira@apache.org>

          Show
          Ted Malaska added a comment - Hey ATM, Sorry for missing that. I updated the code and submitted the new patch. Ted Malaska On Fri, Jun 13, 2014 at 8:11 PM, Aaron T. Myers (JIRA) <jira@apache.org>
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12650460/HADOOP_10610-2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverControllerStress

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4068//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4068//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650460/HADOOP_10610-2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverControllerStress +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/4068//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4068//console This message is automatically generated.
          Hide
          David Chaiken added a comment -

          Thanks very much for fixing this issue. We have run into this problem at Altiscale, and our customers will appreciate the faster performance and higher reliability that we'll be able to implement by specifying multiple S3 buffer directories.

          Show
          David Chaiken added a comment - Thanks very much for fixing this issue. We have run into this problem at Altiscale, and our customers will appreciate the faster performance and higher reliability that we'll be able to implement by specifying multiple S3 buffer directories.
          Hide
          Aaron T. Myers added a comment -

          +1, the latest patch looks good to me. I'm confident that the test failure is unrelated.

          I'm going to commit this momentarily.

          Show
          Aaron T. Myers added a comment - +1, the latest patch looks good to me. I'm confident that the test failure is unrelated. I'm going to commit this momentarily.
          Hide
          Aaron T. Myers added a comment -

          I've just committed this to trunk and branch-2.

          Thanks a lot for the contribution, Ted.

          Show
          Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Ted.
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #5904 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5904/)
          HADOOP-10610. Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #5904 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5904/ ) HADOOP-10610 . Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #616 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/616/)
          HADOOP-10610. Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #616 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/616/ ) HADOOP-10610 . Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1835 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1835/)
          HADOOP-10610. Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1835 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1835/ ) HADOOP-10610 . Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #1808 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1808/)
          HADOOP-10610. Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Show
          Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1808 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1808/ ) HADOOP-10610 . Upgrade S3n s3.fs.buffer.dir to support multi directories. Contributed by Ted Malaska. (atm: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1611489 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3native/NativeS3FileSystem.java
          Hide
          Steve Loughran added a comment -

          I know it is somewhat moot as the code is now checked in, but i am disappointed that my feedback was ignored. In particular: we haven't improved code coverage through new tests, and there's still a constant embedded in the string rather than pulled out for re-use.

          A goal for patches should be to improve the codebase, which means through coverage and other details.

          for this change, in particular, what if the buffer dir is one of

          1. the empty string, ""
          2. a list of directories that are not present/read only.

          I believe the allocator handles this, but tests still matter. Please can people write tests for new features, even things as minor of this. It's a change, and changes need tests

          Show
          Steve Loughran added a comment - I know it is somewhat moot as the code is now checked in, but i am disappointed that my feedback was ignored. In particular: we haven't improved code coverage through new tests, and there's still a constant embedded in the string rather than pulled out for re-use. A goal for patches should be to improve the codebase, which means through coverage and other details. for this change, in particular, what if the buffer dir is one of the empty string, "" a list of directories that are not present/read only. I believe the allocator handles this, but tests still matter. Please can people write tests for new features, even things as minor of this. It's a change, and changes need tests
          Hide
          Ted Malaska added a comment -

          Hey Steve,

          If there is demand for this I would be happy to make it happen. Just let me know. I know that S3a will be sun setting S3n. So if you think this is worth the time I will do it after my latest Spark Jira.

          Ted Malaska

          Show
          Ted Malaska added a comment - Hey Steve, If there is demand for this I would be happy to make it happen. Just let me know. I know that S3a will be sun setting S3n. So if you think this is worth the time I will do it after my latest Spark Jira. Ted Malaska
          Hide
          Aaron T. Myers added a comment -

          Hey Steve - sorry about missing your feedback. This JIRA had been in my backlog to review for a while and I honestly didn't notice your comments - I just opened the JIRA and scrolled to the bottom. Honest mistake.

          Regardless, I think what Ted says makes sense. Seems like S3A will likely supersede S3N, but if you feel strongly we can of course add some tests for what you point out above. I'd be happy to review/commit such a JIRA.

          Show
          Aaron T. Myers added a comment - Hey Steve - sorry about missing your feedback. This JIRA had been in my backlog to review for a while and I honestly didn't notice your comments - I just opened the JIRA and scrolled to the bottom. Honest mistake. Regardless, I think what Ted says makes sense. Seems like S3A will likely supersede S3N, but if you feel strongly we can of course add some tests for what you point out above. I'd be happy to review/commit such a JIRA.
          Hide
          Jason Archip added a comment -

          Could you please update the documentation at https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html to include the options for fs.s3.buffer.dir

          Please let me know if there is a different place to put my request

          Thanks,
          Jason Archip

          Show
          Jason Archip added a comment - Could you please update the documentation at https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html to include the options for fs.s3.buffer.dir Please let me know if there is a different place to put my request Thanks, Jason Archip

            People

            • Assignee:
              Ted Malaska
              Reporter:
              Ted Malaska
            • Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development