Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3518

Provide API to check HDFS operational state

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0, 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy.

      Description

      This will improve the usability of JobTracker safe mode.

      1. h3518_20120611.patch
        7 kB
        Tsz Wo Nicholas Sze
      2. h3518_20120615_b-1.0.patch
        7 kB
        Tsz Wo Nicholas Sze
      3. h3518_20120615.patch
        7 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          Hey Bikas, can you please elaborate a little bit on what you'd like to see out of this API?

          Show
          Aaron T. Myers added a comment - Hey Bikas, can you please elaborate a little bit on what you'd like to see out of this API?
          Hide
          Suresh Srinivas added a comment -

          @Bikas, I have not spent a lot of time on MAPREDUCE-4328. What you want is a mechanism to see the service state of Namenode, so JT can go into safe mode?

          Show
          Suresh Srinivas added a comment - @Bikas, I have not spent a lot of time on MAPREDUCE-4328 . What you want is a mechanism to see the service state of Namenode, so JT can go into safe mode?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          We could add the new API to the org.apache.hadoop.hdfs.client package.

          Show
          Tsz Wo Nicholas Sze added a comment - We could add the new API to the org.apache.hadoop.hdfs.client package.
          Hide
          Bikas Saha added a comment -

          @suresh - Yes
          @atm - Sorry. I thought my title would be sufficient Suresh's comment clarifies what I missed.

          Show
          Bikas Saha added a comment - @suresh - Yes @atm - Sorry. I thought my title would be sufficient Suresh's comment clarifies what I missed.
          Hide
          Aaron T. Myers added a comment -

          @Bikas - my question was more along the lines of what "operational state" you're looking for here. Is the NN up? Is the NN in safe mode? Is the NN "Active" in the HA sense? Something else?

          Show
          Aaron T. Myers added a comment - @Bikas - my question was more along the lines of what "operational state" you're looking for here. Is the NN up? Is the NN in safe mode? Is the NN "Active" in the HA sense? Something else?
          Hide
          Harsh J added a comment -

          Whatever API is being added, be it similar to HDFS-2413 or not, it should help HBase too. Linking HDFS-2413, although I understand that this one is targeting states more on an availability level and just for 1.x apparently. Same may be done for 2.x/trunk too, I imagine several services there benefiting from it too.

          Show
          Harsh J added a comment - Whatever API is being added, be it similar to HDFS-2413 or not, it should help HBase too. Linking HDFS-2413 , although I understand that this one is targeting states more on an availability level and just for 1.x apparently. Same may be done for 2.x/trunk too, I imagine several services there benefiting from it too.
          Hide
          Bikas Saha added a comment -

          Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. HDFS-2413 seems to fit the bill but its committed to the 2.0 line.

          Show
          Bikas Saha added a comment - Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. HDFS-2413 seems to fit the bill but its committed to the 2.0 line.
          Hide
          Aaron T. Myers added a comment -

          Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode.

          Got it. Makes sense. Perhaps then this JIRA should be "back-port HFDS-2413 to branch-1" ?

          HDFS-2413 seems to fit the bill but its committed to the 2.0 line.

          I didn't realize that this JIRA was intended only to address an issue in the branch-1 line, though I should have realized that since the issue description mentions the JT. Mind setting the affects/targets versions?

          Show
          Aaron T. Myers added a comment - Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. Got it. Makes sense. Perhaps then this JIRA should be "back-port HFDS-2413 to branch-1" ? HDFS-2413 seems to fit the bill but its committed to the 2.0 line. I didn't realize that this JIRA was intended only to address an issue in the branch-1 line, though I should have realized that since the issue description mentions the JT. Mind setting the affects/targets versions?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I think this is different from HDFS-2413. Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?

          Show
          Tsz Wo Nicholas Sze added a comment - I think this is different from HDFS-2413 . Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?
          Hide
          Aaron T. Myers added a comment -

          I think this is different from HDFS-2413. Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?

          Seems like the solution to that problem, then, is just to fix HDFS-3507, which would make DFS#isInSafeMode perform client failover.

          Show
          Aaron T. Myers added a comment - I think this is different from HDFS-2413 . Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API? Seems like the solution to that problem, then, is just to fix HDFS-3507 , which would make DFS#isInSafeMode perform client failover.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.

          Show
          Tsz Wo Nicholas Sze added a comment - Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.
          Hide
          Aaron T. Myers added a comment -

          Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.

          Sounds good. Thanks, Nicholas.

          Show
          Aaron T. Myers added a comment - Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API. Sounds good. Thanks, Nicholas.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I still think that isHealthy and isInSafeMode are different:

          • SafeMode is one of the conditions to determine healthy.
          • isInSafeMode may throw exception but isHealthy won't. When there is an exception, isHealthy simply returns false.
          • If retry is set in conf or in a cached DistributedFileSystem, isInSafeMode may retry for a long time. isHealthy disable cache and retry.

          isHealthy is a high level utility method that makes use of isInSafeMode, which is a low level method, so that isHealthy could hide the details of handling cache, retry and exceptions.

          h3518_20120611.patch: adds HdfsUtils.isHealthy(URI).

          Show
          Tsz Wo Nicholas Sze added a comment - I still think that isHealthy and isInSafeMode are different: SafeMode is one of the conditions to determine healthy. isInSafeMode may throw exception but isHealthy won't. When there is an exception, isHealthy simply returns false. If retry is set in conf or in a cached DistributedFileSystem, isInSafeMode may retry for a long time. isHealthy disable cache and retry. isHealthy is a high level utility method that makes use of isInSafeMode, which is a low level method, so that isHealthy could hide the details of handling cache, retry and exceptions. h3518_20120611.patch: adds HdfsUtils.isHealthy(URI).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12531778/h3518_20120611.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531778/h3518_20120611.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Nicholas, patch looks pretty good to me. Two comments:

          1. Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin?
          2. This won't work in an HA setup at the moment since DistributedFileSystem#setSafeMode does not perform client failover, per HDFS-3507. Presumably a user writing against the isHealthy API would like it to return success without considering how many NNs are in play. We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.
          Show
          Aaron T. Myers added a comment - Hey Nicholas, patch looks pretty good to me. Two comments: Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin? This won't work in an HA setup at the moment since DistributedFileSystem#setSafeMode does not perform client failover, per HDFS-3507 . Presumably a user writing against the isHealthy API would like it to return success without considering how many NNs are in play. We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin?

          This is not an admin method. Any user could have an application using it.

          > ... We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.

          It will work for non-HA setup. So, we should not wait for HDFS-3507. If there is a bug in HA, it should not stop other development.

          Show
          Tsz Wo Nicholas Sze added a comment - > Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin? This is not an admin method. Any user could have an application using it. > ... We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended. It will work for non-HA setup. So, we should not wait for HDFS-3507 . If there is a bug in HA, it should not stop other development.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          BTW, the failure of TestFileLengthOnClusterRestart is not related to this.

          Show
          Tsz Wo Nicholas Sze added a comment - BTW, the failure of TestFileLengthOnClusterRestart is not related to this.
          Hide
          Suresh Srinivas added a comment -

          Nicholas, +1 for the patch with following minor comments handled:

          1. Use HDFS_URI_SCHEME instead of hardcoding "hdfs"
          2. Some lines are > 80 chars
          3. HdfsUtils is using in FileSystem.class in LOG
          Show
          Suresh Srinivas added a comment - Nicholas, +1 for the patch with following minor comments handled: Use HDFS_URI_SCHEME instead of hardcoding "hdfs" Some lines are > 80 chars HdfsUtils is using in FileSystem.class in LOG
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3518_20120615.patch:

          • uses HdfsConstants.HDFS_URI_SCHEME in HdfsUtils and also in DistributedFileSystem.getScheme();
          • uses HdfsUtils.class for HdfsUtils.LOG;
          • fixes the line with > 80 character.
          Show
          Tsz Wo Nicholas Sze added a comment - h3518_20120615.patch: uses HdfsConstants.HDFS_URI_SCHEME in HdfsUtils and also in DistributedFileSystem.getScheme(); uses HdfsUtils.class for HdfsUtils.LOG; fixes the line with > 80 character.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks Suresh for the review.

          All changes are minor so that I did not wait for Jenkins.

          I have committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks Suresh for the review. All changes are minor so that I did not wait for Jenkins. I have committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2434/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2434/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2362 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2362/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2362 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2362/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2381 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2381/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2381 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2381/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3518_20120615_b-1.0.patch: for branch 1.0; it requires HDFS-3504 and may not be committed.

          Show
          Tsz Wo Nicholas Sze added a comment - h3518_20120615_b-1.0.patch: for branch 1.0; it requires HDFS-3504 and may not be committed.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Aaron T. Myers added a comment -

          This is not an admin method. Any user could have an application using it.

          I agree it's not an "admin method" in the sense that it doesn't require super-user privileges, but it does seem like an "administrative" utility to me in the sense that it's not something that typical clients need to call, but rather only those clients that are concerned with programmatically administering HDFS. In the class comment in HdfsAdmin, there's no mention of super user privileges:

           * The public API for performing administrative functions on HDFS. Those writing      
           * applications against HDFS should prefer this interface to directly accessing
           * functionality in DistributedFileSystem or DFSClient.
          

          Given all this, I do think "isHealthy" makes more sense in HdfsAdmin than in the new "HdfsUtils" class. Thoughts? I don't mind opening a separate JIRA to discuss this, if you'd prefer.

          Show
          Aaron T. Myers added a comment - This is not an admin method. Any user could have an application using it. I agree it's not an "admin method" in the sense that it doesn't require super-user privileges, but it does seem like an "administrative" utility to me in the sense that it's not something that typical clients need to call, but rather only those clients that are concerned with programmatically administering HDFS. In the class comment in HdfsAdmin, there's no mention of super user privileges: * The public API for performing administrative functions on HDFS. Those writing * applications against HDFS should prefer this interface to directly accessing * functionality in DistributedFileSystem or DFSClient. Given all this, I do think "isHealthy" makes more sense in HdfsAdmin than in the new "HdfsUtils" class. Thoughts? I don't mind opening a separate JIRA to discuss this, if you'd prefer.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Aaron, you have a good point. Let me think about it more and do the move in HDFS-3184 if necessary.

          Show
          Tsz Wo Nicholas Sze added a comment - Aaron, you have a good point. Let me think about it more and do the move in HDFS-3184 if necessary.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The b-1.0 patch also applies to branch-1. Will commit it.

          Show
          Tsz Wo Nicholas Sze added a comment - The b-1.0 patch also applies to branch-1. Will commit it.
          Hide
          Aaron T. Myers added a comment -

          Sounds good, Nicholas. Thanks a lot for hearing me out.

          Show
          Aaron T. Myers added a comment - Sounds good, Nicholas. Thanks a lot for hearing me out.

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development