Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13191

FileSystem#listStatus should not return null

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 2.6.0
    • None
    • fs
    • None

    Description

      This came out of discussion in HADOOP-12718. The FileSystem#listStatus contract does not indicate null is a valid return and some callers do not test null before use:

      AbstractContractGetFileStatusTest#testListStatusEmptyDirectory:

          assertEquals("ls on an empty directory not of length 0", 0,
              fs.listStatus(subfolder).length);
      

      ChecksumFileSystem#copyToLocalFile:

            FileStatus[] srcs = listStatus(src);
            for (FileStatus srcFile : srcs) {
      

      SimpleCopyLIsting#getFileStatus:

            FileStatus[] fileStatuses = fileSystem.listStatus(path);
            if (excludeList != null && excludeList.size() > 0) {
              ArrayList<FileStatus> fileStatusList = new ArrayList<>();
              for(FileStatus status : fileStatuses) {
      

      IMHO, there is no good reason for listStatus to return null. It should throw IOExceptions upon errors or return empty list.

      To enforce the contract that null is an invalid return, update javadoc and leverage @Nullable/@NotNull/@Nonnull annotations.

      So far, I am only aware of the following functions that can return null:

      • RawLocalFileSystem#listStatus

      Attachments

        1. HADOOP-13191.001.patch
          10 kB
          John Zhuge
        2. HADOOP-13191.002.patch
          10 kB
          John Zhuge
        3. HADOOP-13191.003.patch
          12 kB
          John Zhuge
        4. HADOOP-13191.004.patch
          12 kB
          John Zhuge

        Issue Links

          Activity

            People

              jzhuge John Zhuge
              jzhuge John Zhuge
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: