Hadoop Common
  1. Hadoop Common
  2. HADOOP-8606

FileSystem.get may return the wrong filesystem

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 0.23.0, 2.0.0-alpha, 3.0.0
    • Fix Version/s: 1.2.0, 0.23.3, 2.0.2-alpha
    • Component/s: fs
    • Labels:
      None

      Description

      FileSystem.get(URI, conf) will return the default fs if the scheme is null, regardless of whether the authority is null too. This causes URIs of "//authority/path" to always refer to "/path" on the default fs. To the user, this appears to "work" if the authority in the null-scheme URI matches the authority of the default fs. When the authorities don't match, the user is very surprised that the default fs is used.

      1. HADOOP-8606.patch
        4 kB
        Daryn Sharp
      2. HADOOP-8606.branch-1.patch
        3 kB
        Daryn Sharp

        Activity

        Hide
        Matt Foley added a comment -

        added 1.2.0 to fixVersion, per CHANGES.txt.

        Show
        Matt Foley added a comment - added 1.2.0 to fixVersion, per CHANGES.txt.
        Matt Foley made changes -
        Fix Version/s 1.2.0 [ 12321659 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Arun C Murthy made changes -
        Fix Version/s 3.0.0 [ 12320357 ]
        Fix Version/s 1.2.0 [ 12321659 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1147 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1147/)
        HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1147 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1147/ ) HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #324 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/324/)
        svn merge -c 1365224 FIXES: HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365228)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365228
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #324 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/324/ ) svn merge -c 1365224 FIXES: HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365228) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365228 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1115 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1115/)
        HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1115 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1115/ ) HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2537 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2537/)
        HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2537 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2537/ ) HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2516 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2516/)
        HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2516 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2516/ ) HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2581 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2581/)
        HADOOP-8606. FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2581 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2581/ ) HADOOP-8606 . FileSystem.get may return the wrong filesystem (Daryn Sharp via bobby) (Revision 1365224) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1365224 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemCaching.java
        Robert Joseph Evans made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 1.2.0 [ 12321659 ]
        Fix Version/s 0.23.3 [ 12320059 ]
        Fix Version/s 3.0.0 [ 12320357 ]
        Fix Version/s 2.2.0-alpha [ 12322473 ]
        Resolution Fixed [ 1 ]
        Hide
        Robert Joseph Evans added a comment -

        Thanks Daryn,

        I put this into trunk, branch-2, branch-1, and branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Daryn, I put this into trunk, branch-2, branch-1, and branch-0.23
        Hide
        Robert Joseph Evans added a comment -

        I am +1 n this patch too, Thanks Daryn, I'll check this in for you.

        Show
        Robert Joseph Evans added a comment - I am +1 n this patch too, Thanks Daryn, I'll check this in for you.
        Hide
        Suresh Srinivas added a comment -

        +1 for the patch.

        Show
        Suresh Srinivas added a comment - +1 for the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537032/HADOOP-8606.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.ha.TestZKFailoverController

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1204//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1204//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537032/HADOOP-8606.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1204//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1204//console This message is automatically generated.
        Hide
        Daryn Sharp added a comment -

        This happened to be discovered while testing federation (but not related to federation). Old tests using the bad syntax failed when the conf's default fs was changed. Lots of confusion ensued as commands seem to run but the files weren't on the right NN, etc.

        Show
        Daryn Sharp added a comment - This happened to be discovered while testing federation (but not related to federation). Old tests using the bad syntax failed when the conf's default fs was changed. Lots of confusion ensued as commands seem to run but the files weren't on the right NN, etc.
        Daryn Sharp made changes -
        Attachment HADOOP-8606.patch [ 12537032 ]
        Hide
        Daryn Sharp added a comment -

        Resubmitting patch because jenkins only tried to apply the branch-1 patch to trunk...

        Show
        Daryn Sharp added a comment - Resubmitting patch because jenkins only tried to apply the branch-1 patch to trunk...
        Daryn Sharp made changes -
        Attachment HADOOP-8606.patch [ 12537025 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12537026/HADOOP-8606.branch-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1203//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12537026/HADOOP-8606.branch-1.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1203//console This message is automatically generated.
        Daryn Sharp made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Daryn Sharp made changes -
        Field Original Value New Value
        Attachment HADOOP-8606.patch [ 12537025 ]
        Attachment HADOOP-8606.branch-1.patch [ 12537026 ]
        Daryn Sharp created issue -

          People

          • Assignee:
            Daryn Sharp
            Reporter:
            Daryn Sharp
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development