Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-538

DistributedFileSystem::listStatus incorrectly returns null for empty result sets

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1, 0.20.2
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      FileSystem.listStatus() previously returned null for empty or nonexistent directories; will now return empty array for empty directories and throw FileNotFoundException for non-existent directory. Client code should be updated for new semantics.

      Description

      Currently the listStatus method returns null if no files match the request. This differs from the Checksum/LocalFileSystem implementation, which returns an empty array, and the nontvery-explict prescription of the FileSystem interface: "@return the statuses of the files/directories in the given patch" It's better to return an empty collection than have to add extra null checks. The method should return an empty array.

      1. HDFS-538.patch
        2 kB
        Jakob Homan

        Issue Links

          Activity

          Jakob Homan created issue -
          Tom White made changes -
          Field Original Value New Value
          Link This issue is part of HDFS-303 [ HDFS-303 ]
          Jakob Homan made changes -
          Assignee Jakob Homan [ jghoman ]
          Jakob Homan made changes -
          Link This issue relates to HADOOP-3716 [ HADOOP-3716 ]
          Jakob Homan made changes -
          Link This issue is blocked by HADOOP-6201 [ HADOOP-6201 ]
          Jakob Homan made changes -
          Link This issue is depended upon by MAPREDUCE-895 [ MAPREDUCE-895 ]
          Jakob Homan made changes -
          Attachment HDFS-538.patch [ 12417303 ]
          Chris Douglas made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change, Reviewed]
          Fix Version/s 0.21.0 [ 12314046 ]
          Resolution Fixed [ 1 ]
          Jakob Homan made changes -
          Release Note Solidify FileSystem contract for listStatus method when the provided path does not exist. The method is now guaranteed to throw a FileNotFoundException, whereas some FileSystem implementations had previously returned null in this case. Existing code that relies on checking for null should be updated to catch or propagate a FileNotFoundException.
          Jakob Homan made changes -
          Release Note Solidify FileSystem contract for listStatus method when the provided path does not exist. The method is now guaranteed to throw a FileNotFoundException, whereas some FileSystem implementations had previously returned null in this case. Existing code that relies on checking for null should be updated to catch or propagate a FileNotFoundException. Updated DistributedFileSystem::listStatus() method to throw a FileNotFoundException rather than return null in the case that the provided path does not exist. Updated code that relied on checking for null to catch FileNotFoundException. Client code that relied on checking for null should be updated to support the new semantics.
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to HDFS-568 [ HDFS-568 ]
          Robert Chansler made changes -
          Release Note Updated DistributedFileSystem::listStatus() method to throw a FileNotFoundException rather than return null in the case that the provided path does not exist. Updated code that relied on checking for null to catch FileNotFoundException. Client code that relied on checking for null should be updated to support the new semantics.
          Todd Lipcon made changes -
          Release Note FileSystem.listStatus() previously returned null for empty or nonexistent directories. It has been changed to throw FileNotFoundException if the directory does not exist and to return an empty array if the directory is empty.
          Jakob Homan made changes -
          Release Note FileSystem.listStatus() previously returned null for empty or nonexistent directories. It has been changed to throw FileNotFoundException if the directory does not exist and to return an empty array if the directory is empty. FileSystem.listStatus() previously returned null for empty or nonexistent directories; will now return empty array for empty directories and throw FileNotFoundException for non-existent directory. Client code should be updated for new semantics.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Carl Steinbach made changes -
          Affects Version/s 0.20.2 [ 12314204 ]
          Affects Version/s 0.20.1 [ 12314048 ]
          Carl Steinbach made changes -
          Link This issue relates to HIVE-1893 [ HIVE-1893 ]
          Matt Foley made changes -
          Link This issue is related to HADOOP-7352 [ HADOOP-7352 ]

            People

            • Assignee:
              Jakob Homan
              Reporter:
              Jakob Homan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development