Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-960

DistributedFileSystem#makeQualified port inconsistency

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.20.1, 0.20.2, 0.21.0, 0.22.0
    • None
    • hdfs-client
    • None

    Description

      DistributedFileSystem#makeQualified returns paths of form hdfs://host:port/path if the port of the given path does not match the default NN port, and paths of form hdfs://host/path if it does. Is this inentional? It breaks programs that assumes DistributedFileSystem#listStatus paths always have the same form. I think the output of listStatus should not depend on whether the default port is being used. Here's the relevant code:

        /** Normalize paths that explicitly specify the default port. */
        public Path makeQualified(Path path) {
          URI thisUri = this.getUri();
          URI thatUri = path.toUri();
          String thatAuthority = thatUri.getAuthority();
          if (thatUri.getScheme() != null
              && thatUri.getScheme().equalsIgnoreCase(thisUri.getScheme())
              && thatUri.getPort() == NameNode.DEFAULT_PORT
              && thisUri.getPort() == -1
              && thatAuthority.substring(0,thatAuthority.indexOf(":"))
              .equalsIgnoreCase(thisUri.getAuthority())) {
            path = new Path(thisUri.getScheme(), thisUri.getAuthority(),
                            thatUri.getPath());
          }
          return super.makeQualified(path);
        }
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              eli Eli Collins
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: