Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14365

Stabilise FileSystem builder-based create API

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0
    • Fix Version/s: 3.2.0
    • Component/s: fs
    • Labels:
      None
    • Target Version/s:

      Description

      HDFS-11170 added a builder-based create API for file creation which has a few issues to work out before it can be considered ready for use

      1. There no specification in the filesystem.md of what it is meant to do, which means there's no public documentation on expected behaviour except on the Javadocs, which consists of the sentences "Create a new FSDataOutputStreamBuilder for the file with path" and "Base of specific file system FSDataOutputStreamBuilder".

      I propose:

      1. Give the new method a relevant name rather than just define the return type, e.g. createFile().
      2. `Filesystem.md` to be extended with coverage of this method, and, sadly for the authors, coverage of what the semantics of FSDataOutputStreamBuilder.build() are.

      2. There are only tests for HDFS and local, neither of them perfect. Proposed: move to AbstractContractCreateTest, test for all filesystems, fix tests and FS where appropriate.

      3. Add more tests to generate the failure conditions implied by the updated filesystem spec. Eg. create over a an existing file, create over a directory, create with negative buffer size, negative block size, empty dest path, etc, etc.
      This will clarify when precondition checks are made, as well as whether. For example: should newFSDataOutputStreamBuilder() validate the path immediately?

      4. Add to FileContext.

      5. Take the opportunity to look at the flaws in today's create() calls and address them, rather than replicate. In particular, I'd like to end the behaviour "create all parent dirs.

        Attachments

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              eddyxu Lei (Eddy) Xu Assign to me
              Reporter:
              stevel@apache.org Steve Loughran

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment