XMLWordPrintableJSON

    Details

      Description

      The openFile() builder API lets us add new options when reading a file

      Add an option "fs.s3a.open.option.length" which takes a long and allows the length of the file to be declared. If set, no check for the existence of the file is issued when opening the file

      Also: withFileStatus() to take any FileStatus implementation, rather than only S3AFileStatus -and not check that the path matches the path being opened. Needed to support viewFS-style wrapping and mounting.

      and Adopt where appropriate to stop clusters with S3A reads switched to random IO from killing download/localization

      • fs shell copyToLocal
      • distcp
      • IOUtils.copy

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h