Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3483

Better error message when hdfs fsck is run against a ViewFS config

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.5
    • Component/s: None
    • Labels:

      Description

      I'm running a HA + secure + federated cluster.

      When I run "hdfs fsck /nameservices/ha-nn-uri/", I see the following:

      bash-3.2$ hdfs fsck /nameservices/ha-nn-uri/
      FileSystem is viewfs://oracle/
      DFSck exiting.
      Any path I enter will return the same message.

      Attached are my core-site.xml and hdfs-site.xml.

      1. core-site.xml
        3 kB
        Stephen Chu
      2. hdfs-site.xml
        6 kB
        Stephen Chu
      3. HDFS-3483.patch
        1 kB
        Stephen Fritz

        Activity

        Hide
        Stephen Chu added a comment -

        This also impacts other DFSAdmin commands, such as

        bash-3.2$ hdfs dfsadmin -setQuota 10 /nameservices/ha-nn-uri/user/hdfs
        setQuota: FileSystem viewfs://oracle/ is not a distributed file system
        Usage: java DFSAdmin [-setQuota <quota> <dirname>...<dirname>]
        
        Show
        Stephen Chu added a comment - This also impacts other DFSAdmin commands, such as bash-3.2$ hdfs dfsadmin -setQuota 10 /nameservices/ha-nn-uri/user/hdfs setQuota: FileSystem viewfs://oracle/ is not a distributed file system Usage: java DFSAdmin [-setQuota <quota> <dirname>...<dirname>]
        Hide
        Daryn Sharp added a comment -

        As you can tell by the "hdfs" command name, the commands like fsck & dfsadmin are designed specifically for hdfs. With viewfs, it's not possible to know the real fs for a path so the only option is to directly run the commands on the real hdfs filesystem. That's an "easy" answer for fsck, but not for setQuota. Having to know the mount point's fs to run quota commands shatters the transparency of viewfs.

        In either command's case, the required apis are only in DistributedFileSystem. If they were defined by FileSystem to be a no-op or exception, then perhaps there could be a general "hadoop fs -setQuota" or "hadoop fsck". ViewFileSystem could implement the commands to delegate thru to the mounted fs.

        Show
        Daryn Sharp added a comment - As you can tell by the "hdfs" command name, the commands like fsck & dfsadmin are designed specifically for hdfs. With viewfs, it's not possible to know the real fs for a path so the only option is to directly run the commands on the real hdfs filesystem. That's an "easy" answer for fsck, but not for setQuota. Having to know the mount point's fs to run quota commands shatters the transparency of viewfs. In either command's case, the required apis are only in DistributedFileSystem . If they were defined by FileSystem to be a no-op or exception, then perhaps there could be a general "hadoop fs -setQuota" or "hadoop fsck". ViewFileSystem could implement the commands to delegate thru to the mounted fs.
        Hide
        Eli Collins added a comment -

        You could make an argument that fsck should work transparently if the given path is hdfs, eg an admin using a typical federated configuration (where all the file systems are hdfs) might reasonably expect "hdfs fsck /foo/bar" to work.

        We should at least make the error message more friendly, eg like dfsadmin tell the user that the configured file system is viewfs not hdfs. And we should make the dfsadmin command say "not a HDFS file system" instead of "not a distributed file system".

        Show
        Eli Collins added a comment - You could make an argument that fsck should work transparently if the given path is hdfs, eg an admin using a typical federated configuration (where all the file systems are hdfs) might reasonably expect "hdfs fsck /foo/bar" to work. We should at least make the error message more friendly, eg like dfsadmin tell the user that the configured file system is viewfs not hdfs. And we should make the dfsadmin command say "not a HDFS file system" instead of "not a distributed file system".
        Hide
        Daryn Sharp added a comment -

        If I'm understanding correctly, one issue with making fsck work with ViewFileSystem if all underlying filesystems are DFS is that viewfs doesn't expose a way to find the underlying fs for a path. I'm not sure that functionality should be exposed. A possibly clean way is for viewfs/filterfs to do a passthru of a fsck method to the underlying fs.

        I agree that at a minimum the error message should be clearer, not just for viewfs but for any other fs.

        Show
        Daryn Sharp added a comment - If I'm understanding correctly, one issue with making fsck work with ViewFileSystem if all underlying filesystems are DFS is that viewfs doesn't expose a way to find the underlying fs for a path. I'm not sure that functionality should be exposed. A possibly clean way is for viewfs/filterfs to do a passthru of a fsck method to the underlying fs. I agree that at a minimum the error message should be clearer, not just for viewfs but for any other fs.
        Hide
        Robert Joseph Evans added a comment -

        Daryn,

        I tend to disagree that we don't want to expose the mapping. I think it is incredibly useful to be able to know what is happening here, and expose it to the end user so they can then reason about what they want to have happen. For example doing a mv from one federated namespace to another will either be very slow, or it will fail, I don't remember which it is right now. In either case it would be good to expose the mounting to both the end user, and also programatically so that appropriate steps can be taken in those situations. Even if the step is just to call up ops and complain that they have the namespaces all wrong for what they want to do.

        All OSes expose it, type mount on Linux and it will list where each file system is mounted.

        Show
        Robert Joseph Evans added a comment - Daryn, I tend to disagree that we don't want to expose the mapping. I think it is incredibly useful to be able to know what is happening here, and expose it to the end user so they can then reason about what they want to have happen. For example doing a mv from one federated namespace to another will either be very slow, or it will fail, I don't remember which it is right now. In either case it would be good to expose the mounting to both the end user, and also programatically so that appropriate steps can be taken in those situations. Even if the step is just to call up ops and complain that they have the namespaces all wrong for what they want to do. All OSes expose it, type mount on Linux and it will list where each file system is mounted.
        Hide
        Daryn Sharp added a comment -

        Agreed it's nice to see the mounts, but that's what the df command does or should do. We'll have to add a method to the FileSystem base class for ViewFileSystem to override which makes me uneasy.

        Show
        Daryn Sharp added a comment - Agreed it's nice to see the mounts, but that's what the df command does or should do. We'll have to add a method to the FileSystem base class for ViewFileSystem to override which makes me uneasy.
        Hide
        Eli Collins added a comment -

        For now let's just make fsck like dfsadmin tell the user that the configured file system is viewfs not hdfs (and say "not a HDFS file system" instead of "not a distributed file system").

        Bobby, mind filing a follow up jira to track your suggestion?

        Show
        Eli Collins added a comment - For now let's just make fsck like dfsadmin tell the user that the configured file system is viewfs not hdfs (and say "not a HDFS file system" instead of "not a distributed file system"). Bobby, mind filing a follow up jira to track your suggestion?
        Hide
        Stephen Fritz added a comment -

        Changed "a distributed" to "an HDFS" in DFSAdmin.java

        Show
        Stephen Fritz added a comment - Changed "a distributed" to "an HDFS" in DFSAdmin.java
        Hide
        Stephen Fritz added a comment -

        Shouldn't need unit tests as it's just a wording change in a message

        Show
        Stephen Fritz added a comment - Shouldn't need unit tests as it's just a wording change in a message
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12547986/HDFS-3483.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3272//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3272//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12547986/HDFS-3483.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3272//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3272//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        +1, the latest patch looks good to me.

        Unless anyone has further comments, I'm going to go ahead and commit this later today. Per Eli's suggestion, we can track a more thorough solution as Bobby suggests via a separate JIRA.

        Show
        Aaron T. Myers added a comment - +1, the latest patch looks good to me. Unless anyone has further comments, I'm going to go ahead and commit this later today. Per Eli's suggestion, we can track a more thorough solution as Bobby suggests via a separate JIRA.
        Hide
        Aaron T. Myers added a comment -

        I've just committed this to trunk and branch-2.

        Thanks a lot for the contribution, Stephen.

        Show
        Aaron T. Myers added a comment - I've just committed this to trunk and branch-2. Thanks a lot for the contribution, Stephen.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2880 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2880/)
        HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2880 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2880/ ) HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2818 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2818/)
        HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2818 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2818/ ) HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2842 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2842/)
        HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2842 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2842/ ) HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1187 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1187/)
        HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1187 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1187/ ) HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1218 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1218/)
        HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1218 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1218/ ) HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1394864) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1394864 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Hide
        Robert Joseph Evans added a comment -

        I pulled this into branch-0.23 too.

        Show
        Robert Joseph Evans added a comment - I pulled this into branch-0.23 too.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #410 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/410/)
        svn merge -c 1394864 FIXES: HDFS-3483. Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1400218)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1400218
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #410 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/410/ ) svn merge -c 1394864 FIXES: HDFS-3483 . Better error message when hdfs fsck is run against a ViewFS config. Contributed by Stephen Fritz. (Revision 1400218) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1400218 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java

          People

          • Assignee:
            Stephen Fritz
            Reporter:
            Stephen Chu
          • Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development