Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16829 Über-jira: S3A Hadoop 3.4 features
  3. HADOOP-16202

Enhance S3A openFile()

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.0
    • Fix Version/s: None
    • Component/s: fs/s3
    • Labels:
      None

      Description

      The openFile() builder API lets us add new options when reading a file

      Add an option "fs.s3a.open.option.length" which takes a long and allows the length of the file to be declared. If set, no check for the existence of the file is issued when opening the file

      Also: withFileStatus() to take any FileStatus implementation, rather than only S3AFileStatus -and not check that the path matches the path being opened. Needed to support viewFS-style wrapping and mounting.

        Attachments

          Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue HADOOP-16387 S3A openFile() options to allow etag/version to be set

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HADOOP-15625 S3A input stream to use etags/version number to detect changed source files

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HADOOP-16090 S3A Client to add explicit support for versioned stores

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HADOOP-17038 Support positional read in AbfsInputStream

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue HADOOP-16759 Filesystem openFile() builder to take a FileStatus param

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link https://github.com/apache/hadoop/pull/2046

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: