Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3518

Provide API to check HDFS operational state

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0, 2.0.2-alpha
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy.

      Description

      This will improve the usability of JobTracker safe mode.

      1. h3518_20120611.patch
        7 kB
        Tsz Wo Nicholas Sze
      2. h3518_20120615.patch
        7 kB
        Tsz Wo Nicholas Sze
      3. h3518_20120615_b-1.0.patch
        7 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Bikas Saha created issue -
          Bikas Saha made changes -
          Field Original Value New Value
          Link This issue is related to MAPREDUCE-4328 [ MAPREDUCE-4328 ]
          Hide
          Aaron T. Myers added a comment -

          Hey Bikas, can you please elaborate a little bit on what you'd like to see out of this API?

          Show
          Aaron T. Myers added a comment - Hey Bikas, can you please elaborate a little bit on what you'd like to see out of this API?
          Hide
          Suresh Srinivas added a comment -

          @Bikas, I have not spent a lot of time on MAPREDUCE-4328. What you want is a mechanism to see the service state of Namenode, so JT can go into safe mode?

          Show
          Suresh Srinivas added a comment - @Bikas, I have not spent a lot of time on MAPREDUCE-4328 . What you want is a mechanism to see the service state of Namenode, so JT can go into safe mode?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          We could add the new API to the org.apache.hadoop.hdfs.client package.

          Show
          Tsz Wo Nicholas Sze added a comment - We could add the new API to the org.apache.hadoop.hdfs.client package.
          Tsz Wo Nicholas Sze made changes -
          Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
          Component/s hdfs client [ 12312928 ]
          Hide
          Bikas Saha added a comment -

          @suresh - Yes
          @atm - Sorry. I thought my title would be sufficient Suresh's comment clarifies what I missed.

          Show
          Bikas Saha added a comment - @suresh - Yes @atm - Sorry. I thought my title would be sufficient Suresh's comment clarifies what I missed.
          Hide
          Aaron T. Myers added a comment -

          @Bikas - my question was more along the lines of what "operational state" you're looking for here. Is the NN up? Is the NN in safe mode? Is the NN "Active" in the HA sense? Something else?

          Show
          Aaron T. Myers added a comment - @Bikas - my question was more along the lines of what "operational state" you're looking for here. Is the NN up? Is the NN in safe mode? Is the NN "Active" in the HA sense? Something else?
          Harsh J made changes -
          Link This issue supercedes HDFS-2413 [ HDFS-2413 ]
          Hide
          Harsh J added a comment -

          Whatever API is being added, be it similar to HDFS-2413 or not, it should help HBase too. Linking HDFS-2413, although I understand that this one is targeting states more on an availability level and just for 1.x apparently. Same may be done for 2.x/trunk too, I imagine several services there benefiting from it too.

          Show
          Harsh J added a comment - Whatever API is being added, be it similar to HDFS-2413 or not, it should help HBase too. Linking HDFS-2413 , although I understand that this one is targeting states more on an availability level and just for 1.x apparently. Same may be done for 2.x/trunk too, I imagine several services there benefiting from it too.
          Hide
          Bikas Saha added a comment -

          Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. HDFS-2413 seems to fit the bill but its committed to the 2.0 line.

          Show
          Bikas Saha added a comment - Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. HDFS-2413 seems to fit the bill but its committed to the 2.0 line.
          Hide
          Aaron T. Myers added a comment -

          Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode.

          Got it. Makes sense. Perhaps then this JIRA should be "back-port HFDS-2413 to branch-1" ?

          HDFS-2413 seems to fit the bill but its committed to the 2.0 line.

          I didn't realize that this JIRA was intended only to address an issue in the branch-1 line, though I should have realized that since the issue description mentions the JT. Mind setting the affects/targets versions?

          Show
          Aaron T. Myers added a comment - Seems like the most useful information we can get would be if the NN is in safe mode or not because the JT can then go into a conservative mode wrt task failures etc. because these would very likely be related to NN safe mode. Got it. Makes sense. Perhaps then this JIRA should be "back-port HFDS-2413 to branch-1" ? HDFS-2413 seems to fit the bill but its committed to the 2.0 line. I didn't realize that this JIRA was intended only to address an issue in the branch-1 line, though I should have realized that since the issue description mentions the JT. Mind setting the affects/targets versions?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I think this is different from HDFS-2413. Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?

          Show
          Tsz Wo Nicholas Sze added a comment - I think this is different from HDFS-2413 . Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?
          Hide
          Aaron T. Myers added a comment -

          I think this is different from HDFS-2413. Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API?

          Seems like the solution to that problem, then, is just to fix HDFS-3507, which would make DFS#isInSafeMode perform client failover.

          Show
          Aaron T. Myers added a comment - I think this is different from HDFS-2413 . Not in SafeMode is necessary condition but not a sufficient condition of being operational. For example, if a namenode is standby but not in safemode, it is not operational. How about add a new method, say isHealth(URI), as a public API? Seems like the solution to that problem, then, is just to fix HDFS-3507 , which would make DFS#isInSafeMode perform client failover.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.

          Show
          Tsz Wo Nicholas Sze added a comment - Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.
          Hide
          Aaron T. Myers added a comment -

          Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API.

          Sounds good. Thanks, Nicholas.

          Show
          Aaron T. Myers added a comment - Hi Aaron, you are right that HDFS-3507 and this is closely related. I will think about if we still need the isHealth(URI) API. Sounds good. Thanks, Nicholas.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I still think that isHealthy and isInSafeMode are different:

          • SafeMode is one of the conditions to determine healthy.
          • isInSafeMode may throw exception but isHealthy won't. When there is an exception, isHealthy simply returns false.
          • If retry is set in conf or in a cached DistributedFileSystem, isInSafeMode may retry for a long time. isHealthy disable cache and retry.

          isHealthy is a high level utility method that makes use of isInSafeMode, which is a low level method, so that isHealthy could hide the details of handling cache, retry and exceptions.

          h3518_20120611.patch: adds HdfsUtils.isHealthy(URI).

          Show
          Tsz Wo Nicholas Sze added a comment - I still think that isHealthy and isInSafeMode are different: SafeMode is one of the conditions to determine healthy. isInSafeMode may throw exception but isHealthy won't. When there is an exception, isHealthy simply returns false. If retry is set in conf or in a cached DistributedFileSystem, isInSafeMode may retry for a long time. isHealthy disable cache and retry. isHealthy is a high level utility method that makes use of isInSafeMode, which is a low level method, so that isHealthy could hide the details of handling cache, retry and exceptions. h3518_20120611.patch: adds HdfsUtils.isHealthy(URI).
          Tsz Wo Nicholas Sze made changes -
          Attachment h3518_20120611.patch [ 12531778 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12531778/h3518_20120611.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531778/h3518_20120611.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestFileLengthOnClusterRestart +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2642//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          Hey Nicholas, patch looks pretty good to me. Two comments:

          1. Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin?
          2. This won't work in an HA setup at the moment since DistributedFileSystem#setSafeMode does not perform client failover, per HDFS-3507. Presumably a user writing against the isHealthy API would like it to return success without considering how many NNs are in play. We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.
          Show
          Aaron T. Myers added a comment - Hey Nicholas, patch looks pretty good to me. Two comments: Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin? This won't work in an HA setup at the moment since DistributedFileSystem#setSafeMode does not perform client failover, per HDFS-3507 . Presumably a user writing against the isHealthy API would like it to return success without considering how many NNs are in play. We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin?

          This is not an admin method. Any user could have an application using it.

          > ... We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended.

          It will work for non-HA setup. So, we should not wait for HDFS-3507. If there is a bug in HA, it should not stop other development.

          Show
          Tsz Wo Nicholas Sze added a comment - > Perhaps this method could reasonably be added to o.a.h.hdfs.client.HdfsAdmin? This is not an admin method. Any user could have an application using it. > ... We might want to wait for resolution of that issue before we introduce this new API that we know won't work as intended. It will work for non-HA setup. So, we should not wait for HDFS-3507 . If there is a bug in HA, it should not stop other development.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          BTW, the failure of TestFileLengthOnClusterRestart is not related to this.

          Show
          Tsz Wo Nicholas Sze added a comment - BTW, the failure of TestFileLengthOnClusterRestart is not related to this.
          Hide
          Suresh Srinivas added a comment -

          Nicholas, +1 for the patch with following minor comments handled:

          1. Use HDFS_URI_SCHEME instead of hardcoding "hdfs"
          2. Some lines are > 80 chars
          3. HdfsUtils is using in FileSystem.class in LOG
          Show
          Suresh Srinivas added a comment - Nicholas, +1 for the patch with following minor comments handled: Use HDFS_URI_SCHEME instead of hardcoding "hdfs" Some lines are > 80 chars HdfsUtils is using in FileSystem.class in LOG
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3518_20120615.patch:

          • uses HdfsConstants.HDFS_URI_SCHEME in HdfsUtils and also in DistributedFileSystem.getScheme();
          • uses HdfsUtils.class for HdfsUtils.LOG;
          • fixes the line with > 80 character.
          Show
          Tsz Wo Nicholas Sze added a comment - h3518_20120615.patch: uses HdfsConstants.HDFS_URI_SCHEME in HdfsUtils and also in DistributedFileSystem.getScheme(); uses HdfsUtils.class for HdfsUtils.LOG; fixes the line with > 80 character.
          Tsz Wo Nicholas Sze made changes -
          Attachment h3518_20120615.patch [ 12532256 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks Suresh for the review.

          All changes are minor so that I did not wait for Jenkins.

          I have committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks Suresh for the review. All changes are minor so that I did not wait for Jenkins. I have committed this.
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Release Note Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy.
          Fix Version/s 2.0.1-alpha [ 12321440 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2434/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2434 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2434/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2362 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2362/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2362 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2362/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2381 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2381/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2381 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2381/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3518_20120615_b-1.0.patch: for branch 1.0; it requires HDFS-3504 and may not be committed.

          Show
          Tsz Wo Nicholas Sze added a comment - h3518_20120615_b-1.0.patch: for branch 1.0; it requires HDFS-3504 and may not be committed.
          Tsz Wo Nicholas Sze made changes -
          Attachment h3518_20120615_b-1.0.patch [ 12532279 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = SUCCESS
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1078 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1078/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/)
          HDFS-3518. Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825)

          Result = FAILURE
          szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1111/ ) HDFS-3518 . Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. (Revision 1350825) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1350825 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/client/HdfsUtils.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientRetries.java
          Hide
          Aaron T. Myers added a comment -

          This is not an admin method. Any user could have an application using it.

          I agree it's not an "admin method" in the sense that it doesn't require super-user privileges, but it does seem like an "administrative" utility to me in the sense that it's not something that typical clients need to call, but rather only those clients that are concerned with programmatically administering HDFS. In the class comment in HdfsAdmin, there's no mention of super user privileges:

           * The public API for performing administrative functions on HDFS. Those writing      
           * applications against HDFS should prefer this interface to directly accessing
           * functionality in DistributedFileSystem or DFSClient.
          

          Given all this, I do think "isHealthy" makes more sense in HdfsAdmin than in the new "HdfsUtils" class. Thoughts? I don't mind opening a separate JIRA to discuss this, if you'd prefer.

          Show
          Aaron T. Myers added a comment - This is not an admin method. Any user could have an application using it. I agree it's not an "admin method" in the sense that it doesn't require super-user privileges, but it does seem like an "administrative" utility to me in the sense that it's not something that typical clients need to call, but rather only those clients that are concerned with programmatically administering HDFS. In the class comment in HdfsAdmin, there's no mention of super user privileges: * The public API for performing administrative functions on HDFS. Those writing * applications against HDFS should prefer this interface to directly accessing * functionality in DistributedFileSystem or DFSClient. Given all this, I do think "isHealthy" makes more sense in HdfsAdmin than in the new "HdfsUtils" class. Thoughts? I don't mind opening a separate JIRA to discuss this, if you'd prefer.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Aaron, you have a good point. Let me think about it more and do the move in HDFS-3184 if necessary.

          Show
          Tsz Wo Nicholas Sze added a comment - Aaron, you have a good point. Let me think about it more and do the move in HDFS-3184 if necessary.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          The b-1.0 patch also applies to branch-1. Will commit it.

          Show
          Tsz Wo Nicholas Sze added a comment - The b-1.0 patch also applies to branch-1. Will commit it.
          Tsz Wo Nicholas Sze made changes -
          Fix Version/s 1.1.0 [ 12317959 ]
          Hide
          Aaron T. Myers added a comment -

          Sounds good, Nicholas. Thanks a lot for hearing me out.

          Show
          Aaron T. Myers added a comment - Sounds good, Nicholas. Thanks a lot for hearing me out.
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          taojie made changes -
          Description This will improve the usability of JobTracker safe mode. This will improve the usability of JobTracker safe mode.得到的
          taojie made changes -
          Description This will improve the usability of JobTracker safe mode.得到的 This will improve the usability of JobTracker safe mode.
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          3d 9h 1 Tsz Wo Nicholas Sze 12/Jun/12 07:33
          Patch Available Patch Available Resolved Resolved
          3d 16h 30m 1 Tsz Wo Nicholas Sze 16/Jun/12 00:04
          Resolved Resolved Closed Closed
          117d 18h 42m 1 Arun C Murthy 11/Oct/12 18:46

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development