Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13191

FileSystem#listStatus should not return null

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 2.6.0
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      This came out of discussion in HADOOP-12718. The FileSystem#listStatus contract does not indicate null is a valid return and some callers do not test null before use:

      AbstractContractGetFileStatusTest#testListStatusEmptyDirectory:

          assertEquals("ls on an empty directory not of length 0", 0,
              fs.listStatus(subfolder).length);
      

      ChecksumFileSystem#copyToLocalFile:

            FileStatus[] srcs = listStatus(src);
            for (FileStatus srcFile : srcs) {
      

      SimpleCopyLIsting#getFileStatus:

            FileStatus[] fileStatuses = fileSystem.listStatus(path);
            if (excludeList != null && excludeList.size() > 0) {
              ArrayList<FileStatus> fileStatusList = new ArrayList<>();
              for(FileStatus status : fileStatuses) {
      

      IMHO, there is no good reason for listStatus to return null. It should throw IOExceptions upon errors or return empty list.

      To enforce the contract that null is an invalid return, update javadoc and leverage @Nullable/@NotNull/@Nonnull annotations.

      So far, I am only aware of the following functions that can return null:

      • RawLocalFileSystem#listStatus

        Attachments

        1. HADOOP-13191.004.patch
          12 kB
          John Zhuge
        2. HADOOP-13191.003.patch
          12 kB
          John Zhuge
        3. HADOOP-13191.002.patch
          10 kB
          John Zhuge
        4. HADOOP-13191.001.patch
          10 kB
          John Zhuge

          Issue Links

            Activity

              People

              • Assignee:
                jzhuge John Zhuge
                Reporter:
                jzhuge John Zhuge
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: