Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12009

Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      FileSystem.listStatus does not guarantee that implementations will return sorted entries:

        /**
         * List the statuses of the files/directories in the given path if the path is
         * a directory.
         * 
         * @param f given path
         * @return the statuses of the files/directories in the given patch
         * @throws FileNotFoundException when the path does not exist;
         *         IOException see specific implementation
         */
        public abstract FileStatus[] listStatus(Path f) throws FileNotFoundException, 
                                                               IOException;

      However, FileSystemContractBaseTest, expects the elements to come back sorted:

          Path[] testDirs = { path("/test/hadoop/a"),
                              path("/test/hadoop/b"),
                              path("/test/hadoop/c/1"), };
         
          // ...
      
          paths = fs.listStatus(path("/test/hadoop"));
          assertEquals(3, paths.length);
          assertEquals(path("/test/hadoop/a"), paths[0].getPath());
          assertEquals(path("/test/hadoop/b"), paths[1].getPath());
          assertEquals(path("/test/hadoop/c"), paths[2].getPath());

      We should pass this test as long as all the paths are there, regardless of their ordering.

      Attachments

        1. HADOOP-12009.1.patch
          4 kB
          J.Andreina
        2. HADOOP-12009-003.patch
          5 kB
          Steve Loughran
        3. HADOOP-12009-004.patch
          5 kB
          J.Andreina

        Issue Links

          Activity

            People

              andreina J.Andreina
              jghoman Jakob Homan
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: