Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12991

Conflicting default ports in DelegateToFileSystem

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.2
    • 2.8.0, 2.7.4, 3.0.0-alpha1
    • fs
    • None
    • Reviewed

    Description

      HADOOP-12304 introduced logic to ensure that the DelegateToFileSystem constructor sets the default port to -1:

      DelegateToFileSystem.java
        protected DelegateToFileSystem(URI theUri, FileSystem theFsImpl,
            Configuration conf, String supportedScheme, boolean authorityRequired)
            throws IOException, URISyntaxException {
          super(theUri, supportedScheme, authorityRequired, 
              getDefaultPortIfDefined(theFsImpl));
          fsImpl = theFsImpl;
          fsImpl.initialize(theUri, conf);
          fsImpl.statistics = getStatistics();
        }
      
        private static int getDefaultPortIfDefined(FileSystem theFsImpl) {
          int defaultPort = theFsImpl.getDefaultPort();
          return defaultPort != 0 ? defaultPort : -1;
        }
      

      However, DelegateToFileSystem#getUriDefaultPort returns 0:

      DelegateToFileSystem.java
        public int getUriDefaultPort() {
          return 0;
        }
      

      This breaks AbstractFileSystem#checkPath:

      AbstractFileSystem.java
          int thisPort = this.getUri().getPort(); // If using DelegateToFileSystem, this is -1
          int thatPort = uri.getPort(); // This is -1 by default in java.net.URI
          if (thatPort == -1) {
            thatPort = this.getUriDefaultPort();  // Sets thatPort to 0
          }
          if (thisPort != thatPort) {
            throw new InvalidPathException("Wrong FS: " + path + ", expected: "
                + this.getUri());
          }
      

      Which breaks any subclasses of DelegateToFileSystem that don't specify a port.

      Attachments

        1. HADOOP-12991.03.patch
          5 kB
          Kai
        2. HADOOP-12991.02.patch
          5 kB
          Kai
        3. HADOOP-12991.01.patch
          5 kB
          Kai

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            lewuathe Kai
            hogeland Kevin Hogeland
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment