Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-729

fsck option to list only corrupted files

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      An option to fsck to list only corrupted files will be very helpful for frequent monitoring.

      1. HDFS-729.6.patch
        18 kB
        Rodrigo Schmidt
      2. HDFS-729.5.patch
        18 kB
        Rodrigo Schmidt
      3. HDFS-729.4.patch
        18 kB
        Rodrigo Schmidt
      4. HDFS-729.3.patch
        18 kB
        Rodrigo Schmidt
      5. HDFS-729.2.patch
        15 kB
        Rodrigo Schmidt
      6. HDFS-729.1.patch
        9 kB
        Rodrigo Schmidt
      7. corruptFiles.txt
        17 kB
        dhruba borthakur
      8. badFiles2.txt
        20 kB
        dhruba borthakur
      9. badFiles.txt
        19 kB
        dhruba borthakur

        Issue Links

          Activity

          Hide
          dhruba borthakur added a comment -

          There are two existing options to handle corrupted files, one option moves the file to lost+found and the other option deletes the corrupted file. I would like to add another option "listCorruptedFiles" that will list the corrupted files if any. An alternative is to running a "fsck -files" and then filter the output on the client to display only corrupted files; but on a cluster with 20 million files, the total amount of data (one for every line of output) to be transferred to the client is huge and introduces lots of latency.

          Show
          dhruba borthakur added a comment - There are two existing options to handle corrupted files, one option moves the file to lost+found and the other option deletes the corrupted file. I would like to add another option "listCorruptedFiles" that will list the corrupted files if any. An alternative is to running a "fsck -files" and then filter the output on the client to display only corrupted files; but on a cluster with 20 million files, the total amount of data (one for every line of output) to be transferred to the client is huge and introduces lots of latency.
          Hide
          Allen Wittenauer added a comment -

          Love this idea. +1

          Show
          Allen Wittenauer added a comment - Love this idea. +1
          Hide
          Raghu Angadi added a comment -

          Is this a regular fsck with less output? That might still be prohibitively long and expensive for regular poll.

          Name node already has a list of all the corrupt/missing/underreplicated blocks. It might be better to list those blocks and files they belong to (a user friendly -metaSave). Would be nice to have jsp as well.

          Show
          Raghu Angadi added a comment - Is this a regular fsck with less output? That might still be prohibitively long and expensive for regular poll. Name node already has a list of all the corrupt/missing/underreplicated blocks. It might be better to list those blocks and files they belong to (a user friendly -metaSave). Would be nice to have jsp as well.
          Hide
          Allen Wittenauer added a comment -

          Using -metaSave as the "tell me what is broke" option is not obvious. (I'm not even user what -metaSave is supposed to mean!) I'd rather have '-listCorruptedFiles' and '-listCorruptedBlocks' or a -listCorrupted that takes an option of files or blocks with the default being files.

          [Random rant: if I had a time machine, I'd like to go back and take the keyboard away from the person who decided camelcase for shell options was OK. Even if this means it is Doug. :) ]

          Show
          Allen Wittenauer added a comment - Using -metaSave as the "tell me what is broke" option is not obvious. (I'm not even user what -metaSave is supposed to mean!) I'd rather have '-listCorruptedFiles' and '-listCorruptedBlocks' or a -listCorrupted that takes an option of files or blocks with the default being files. [Random rant: if I had a time machine, I'd like to go back and take the keyboard away from the person who decided camelcase for shell options was OK. Even if this means it is Doug. :) ]
          Hide
          Raghu Angadi added a comment -

          > Using -metaSave as the "tell me what is broke" option is not obvious. (I'm not even user what -metaSave is supposed to mean!) I'd rather have '-listCorruptedFiles' and '-listCorruptedBlocks' or a -listCorrupted that takes an option of files or blocks with the default being files.

          yes. this is what I meant. metaSave was mentioned only as a quick reference (for a few that used it in the past).

          Show
          Raghu Angadi added a comment - > Using -metaSave as the "tell me what is broke" option is not obvious. (I'm not even user what -metaSave is supposed to mean!) I'd rather have '-listCorruptedFiles' and '-listCorruptedBlocks' or a -listCorrupted that takes an option of files or blocks with the default being files. yes. this is what I meant. metaSave was mentioned only as a quick reference (for a few that used it in the past).
          Hide
          dhruba borthakur added a comment -

          > Is this a regular fsck with less output? That might still be prohibitively long and expensive for regular poll

          Yes, at this point, I am visualizing it as a regular fsck with less output. The problem with making this a new Namenode RPC is that this RPC would have an upper limit on the number of corrupted files that can be returned via one single invocation of the RPC. This kind-of- reduces the elegance of such an API. The alternative is to make this new RPC retrieve a max number of corrupted files together with a cookie that can be used in the next invocation of the RPC to retrieve the remaining set of corrupted files (similar to readdir).

          If we use a regular fsck, it does not lock the NN for an extended period of time, neither does it have a problem if the number of files to be retrieved is huge.

          Show
          dhruba borthakur added a comment - > Is this a regular fsck with less output? That might still be prohibitively long and expensive for regular poll Yes, at this point, I am visualizing it as a regular fsck with less output. The problem with making this a new Namenode RPC is that this RPC would have an upper limit on the number of corrupted files that can be returned via one single invocation of the RPC. This kind-of- reduces the elegance of such an API. The alternative is to make this new RPC retrieve a max number of corrupted files together with a cookie that can be used in the next invocation of the RPC to retrieve the remaining set of corrupted files (similar to readdir). If we use a regular fsck, it does not lock the NN for an extended period of time, neither does it have a problem if the number of files to be retrieved is huge.
          Hide
          Koji Noguchi added a comment -

          Name node already has a list of all the corrupt/missing/underreplicated blocks.

          Fsck is useful when Namenode is misbehaving, like not replicating blocks, not deleting over-replicated blocks, etc.
          In those cases, Namenode's state is usually out of sync and not that useful.

          Show
          Koji Noguchi added a comment - Name node already has a list of all the corrupt/missing/underreplicated blocks. Fsck is useful when Namenode is misbehaving, like not replicating blocks, not deleting over-replicated blocks, etc. In those cases, Namenode's state is usually out of sync and not that useful.
          Hide
          dhruba borthakur added a comment -

          I am planning to follow Raghu's advice and add the following API to the namenode:

          /**

          • Returns a list of files that are corrupted.
          • <p>
          • Returns a list of files that have at least one block that has no valid replicas.
          • The returned list has numExpectedFiles files in it. If the number of files
          • returned is smaller than numExpectedFiles, then it implies that no more
          • corrupted files are available in the system. The startingNumber is the
          • startingNumber-th corrupted file in the system.
          • @param numExpectedFiles the maximum number of files to be returned
          • @param startingNumber list files starting from startingNumberth to
          • (startingNumber + numExpectedFiles)th in the
          • list of corrupted files
          • @throws AccessControlException if the superuser privilege is violated.
          • @throws IOException if unable to retrieve information of a corrupt file
            */
            public LocatedBlocks[] getCorruptFiles(int numExpectedFiles, int startingNumber) throws IOException;

          This will be used by fsck (or any other application) to quickly detect corrupted files.

          Show
          dhruba borthakur added a comment - I am planning to follow Raghu's advice and add the following API to the namenode: /** Returns a list of files that are corrupted. <p> Returns a list of files that have at least one block that has no valid replicas. The returned list has numExpectedFiles files in it. If the number of files returned is smaller than numExpectedFiles, then it implies that no more corrupted files are available in the system. The startingNumber is the startingNumber-th corrupted file in the system. @param numExpectedFiles the maximum number of files to be returned @param startingNumber list files starting from startingNumberth to (startingNumber + numExpectedFiles)th in the list of corrupted files @throws AccessControlException if the superuser privilege is violated. @throws IOException if unable to retrieve information of a corrupt file */ public LocatedBlocks[] getCorruptFiles(int numExpectedFiles, int startingNumber) throws IOException; This will be used by fsck (or any other application) to quickly detect corrupted files.
          Hide
          dhruba borthakur added a comment -

          This introduces a new method to ClientProtocol to retrive the list of corrupted files from the namenode. The server restricts that only 100 files can be retrieved by one invocation of this call. The application using this API has to iteratively call this method multiple times if it wants to retrieve all corrupted files. Only a superuser can invoke this call. Here is the javadoc:

          
            /**
             * Returns a list of files that are corrupted.
             * <p>
             * Returns a list of files that have at least one block that has no valid replicas.
             * The returned list has numExpectedFiles files in it. If the number of files
             * returned is zero, then it implies that no more 
             * corrupted files are available in the system. The startingNumber is the 
             * startingNumber-th corrupted file in the system. 
             * An application will typicaly invoke this method as
             *   int startingNumber = 0;
             *   LocatedBlocks[] l = getCorruptFiles(500, startingNumber);
             *   while (l.size() > 0) {
             *     while (LocatedBlocks onefile: l) {
             *       processOneCorruptedFile(onefile);
             *     }
             *     startingNumber += l.size();
             *     l = getCorruptFiles(100, startingNumber);
             *   }
          
             * 
             * @param numExpectedFiles the maximum number of files to be returned 
             * @param startingNumber list files starting from startingNumberth to 
             *                       (startingNumber + numExpectedFiles)th in the 
             *                       list of corrupted files
             * @throws AccessControlException if the superuser privilege is violated.
             * @throws IOException if unable to retrieve information of a corrupt file
          
          

          I am in the process of writing a unit test for this one.

          Show
          dhruba borthakur added a comment - This introduces a new method to ClientProtocol to retrive the list of corrupted files from the namenode. The server restricts that only 100 files can be retrieved by one invocation of this call. The application using this API has to iteratively call this method multiple times if it wants to retrieve all corrupted files. Only a superuser can invoke this call. Here is the javadoc: /** * Returns a list of files that are corrupted. * <p> * Returns a list of files that have at least one block that has no valid replicas. * The returned list has numExpectedFiles files in it. If the number of files * returned is zero, then it implies that no more * corrupted files are available in the system. The startingNumber is the * startingNumber-th corrupted file in the system. * An application will typicaly invoke this method as * int startingNumber = 0; * LocatedBlocks[] l = getCorruptFiles(500, startingNumber); * while (l.size() > 0) { * while (LocatedBlocks onefile: l) { * processOneCorruptedFile(onefile); * } * startingNumber += l.size(); * l = getCorruptFiles(100, startingNumber); * } * * @param numExpectedFiles the maximum number of files to be returned * @param startingNumber list files starting from startingNumberth to * (startingNumber + numExpectedFiles)th in the * list of corrupted files * @ throws AccessControlException if the superuser privilege is violated. * @ throws IOException if unable to retrieve information of a corrupt file I am in the process of writing a unit test for this one.
          Hide
          dhruba borthakur added a comment -

          This patch allows an API to retrieve the list of files that have blocks with no valid replicas. The blocks either have no replicas or all of their existing replicas are corrupted. This API can be used by fsck/jsp to quickly list files that need attention.

          A new call ClientProtocol.getBadFiles() is introduced. It returns a specified number of files per invocation. It can be used by an application to iterate over all files that have bad blocks.

          Show
          dhruba borthakur added a comment - This patch allows an API to retrieve the list of files that have blocks with no valid replicas. The blocks either have no replicas or all of their existing replicas are corrupted. This API can be used by fsck/jsp to quickly list files that need attention. A new call ClientProtocol.getBadFiles() is introduced. It returns a specified number of files per invocation. It can be used by an application to iterate over all files that have bad blocks.
          Hide
          dhruba borthakur added a comment -

          merged patch with latest trunk

          Show
          dhruba borthakur added a comment - merged patch with latest trunk
          Hide
          Raghu Angadi added a comment -

          Hi Dhruba,

          Patch looks good. What would be an alternative for "badFile"? A "corruptFile", might not imply "one of the blocks no good replica". But in general, "corrupt file" implies something that could not be recovered by filesystem. Between these two, my vote is for "corruptFile".

          The current API is fine : one minor nit is that even when it returns less than numExpectedFiles it does not imply there aren't any more.
          In practice, it is probably good enough not to have this limit and always return up to (100 or 500 files).. This would simplify the interface. Your choice.

          I will double check if all the blocks in 'pri 2' bucket includes all the and only the blocks with no good replica left.

          Show
          Raghu Angadi added a comment - Hi Dhruba, Patch looks good. What would be an alternative for "badFile"? A "corruptFile", might not imply "one of the blocks no good replica". But in general, "corrupt file" implies something that could not be recovered by filesystem. Between these two, my vote is for "corruptFile". The current API is fine : one minor nit is that even when it returns less than numExpectedFiles it does not imply there aren't any more. In practice, it is probably good enough not to have this limit and always return up to (100 or 500 files).. This would simplify the interface. Your choice. I will double check if all the blocks in 'pri 2' bucket includes all the and only the blocks with no good replica left.
          Hide
          Hairong Kuang added a comment -

          I have two concerns about the approach:
          1. If the neededReplicationQueue changes while issuing getBadFiles calls, consecutive calls may not be able to return all bad files;
          2. Because neededReplicationQueue stores blocks that may belong to the same file, so two consecutive badFiles calls may contain duplicate files.

          Show
          Hairong Kuang added a comment - I have two concerns about the approach: 1. If the neededReplicationQueue changes while issuing getBadFiles calls, consecutive calls may not be able to return all bad files; 2. Because neededReplicationQueue stores blocks that may belong to the same file, so two consecutive badFiles calls may contain duplicate files.
          Hide
          dhruba borthakur added a comment -

          Hi Hairong and Raghu, I had a discussion with Konstantin and he suggested simplifying the API by making this API always return the first 500 corrupted files. Even if there are more than 500 corrupted files in the system, successive calls to this API will always return only 500 files. There is no relationship between the files returned by one invocation of this API with the ones returned by the next invocation of this API.

          The above proposal address Hairongs (1) above. It also is in line with Raghu's suggestion "t is probably good enough to always return up to (100 or 500 files". Regarding Hairong's (2) above, this should be addressed and will fixed in the next patch (if not already there).

          Does this sound like a reasonable approach to follow?

          Show
          dhruba borthakur added a comment - Hi Hairong and Raghu, I had a discussion with Konstantin and he suggested simplifying the API by making this API always return the first 500 corrupted files. Even if there are more than 500 corrupted files in the system, successive calls to this API will always return only 500 files. There is no relationship between the files returned by one invocation of this API with the ones returned by the next invocation of this API. The above proposal address Hairongs (1) above. It also is in line with Raghu's suggestion "t is probably good enough to always return up to (100 or 500 files". Regarding Hairong's (2) above, this should be addressed and will fixed in the next patch (if not already there). Does this sound like a reasonable approach to follow?
          Hide
          Raghu Angadi added a comment -

          > [...] simplifying the API by making this API always return the first 500 corrupted files. [...]
          +1.

          Show
          Raghu Angadi added a comment - > [...] simplifying the API by making this API always return the first 500 corrupted files. [...] +1.
          Hide
          Allen Wittenauer added a comment -

          Hmm. So basically the burden on removing duplicates is passed on to the caller? How does the caller know that it has them all?

          Show
          Allen Wittenauer added a comment - Hmm. So basically the burden on removing duplicates is passed on to the caller? How does the caller know that it has them all?
          Hide
          dhruba borthakur added a comment -

          > o basically the burden on removing duplicates is passed on to the c

          This is correct. fsck will actually invoke this API only once and will print out the list of first 500 corrupted files. It will not invoke this API multiple times. This list actually helps the adminstrator because he/she can get a partial list of corrupted files very quickly. My theory is that this partial list is better than waiting for a total fsck to finish which can take hours.

          Show
          dhruba borthakur added a comment - > o basically the burden on removing duplicates is passed on to the c This is correct. fsck will actually invoke this API only once and will print out the list of first 500 corrupted files. It will not invoke this API multiple times. This list actually helps the adminstrator because he/she can get a partial list of corrupted files very quickly. My theory is that this partial list is better than waiting for a total fsck to finish which can take hours.
          Hide
          Raghu Angadi added a comment -

          As I understand, list of files returned in one call will not have duplicates.

          500 is a lot.. note that these are files with 'hard-corruption', ie, HDFS could not repair them. Once a cluster has so many corrupt files, I would think there would be a lot more urgent things to worry about than finding rest of the corrupt files. In practice, most likely reason for such a scenario would be a large number of datanodes go missing.

          Show
          Raghu Angadi added a comment - As I understand, list of files returned in one call will not have duplicates. 500 is a lot.. note that these are files with 'hard-corruption', ie, HDFS could not repair them. Once a cluster has so many corrupt files, I would think there would be a lot more urgent things to worry about than finding rest of the corrupt files. In practice, most likely reason for such a scenario would be a large number of datanodes go missing.
          Hide
          Rodrigo Schmidt added a comment -

          Returning a quick list of 500 corrupted files is a great idea. Most of the time, this will cover all corrupted files in the system.

          +1 to this idea.

          Show
          Rodrigo Schmidt added a comment - Returning a quick list of 500 corrupted files is a great idea. Most of the time, this will cover all corrupted files in the system. +1 to this idea.
          Hide
          dhruba borthakur added a comment -

          It appears that I have consensus from Raghu, Konstantin and Rodrigo. anybody else wants to weigh on in this one?

          Show
          dhruba borthakur added a comment - It appears that I have consensus from Raghu, Konstantin and Rodrigo. anybody else wants to weigh on in this one?
          Hide
          Rodrigo Schmidt added a comment -

          I was looking at the current patch and I think I found a bug on it.

          On UnderReplicatedBlocks.java, the following method was added:

          + /**
          + * Return an iterator of all blocks that have no valid replicas.
          + * These are either blocks with no replicas or all existing replicas
          + * are corrupted. Such blocks are at level 2.
          + */
          + public synchronized Iterator<Block> iteratorBadBlocks()

          { + return priorityQueues.get(2).iterator(); + }

          It assumes all blocks on queue 2 have 0 replicas. However according to get getPriority() on the same source file, we can see that level 2 is also used for blocks whose number of replicas times 3 is bigger than expected replicas:

          } else if(curReplicas*3<expectedReplicas)

          { return 1; }

          else

          { return 2; }

          So, if a block has 2 replicas, but it is expected to have 3, it will also be kept in the queue with priority 2. I'm fixing that by adding an extra check on the real number of replicas a block has before adding it to the list returned by BlockManager.getCorruptFiles() (previously BockManager.getBadFiles()). Please let me know what you guys think about it.

          Show
          Rodrigo Schmidt added a comment - I was looking at the current patch and I think I found a bug on it. On UnderReplicatedBlocks.java, the following method was added: + /** + * Return an iterator of all blocks that have no valid replicas. + * These are either blocks with no replicas or all existing replicas + * are corrupted. Such blocks are at level 2. + */ + public synchronized Iterator<Block> iteratorBadBlocks() { + return priorityQueues.get(2).iterator(); + } It assumes all blocks on queue 2 have 0 replicas. However according to get getPriority() on the same source file, we can see that level 2 is also used for blocks whose number of replicas times 3 is bigger than expected replicas: } else if(curReplicas*3<expectedReplicas) { return 1; } else { return 2; } So, if a block has 2 replicas, but it is expected to have 3, it will also be kept in the queue with priority 2. I'm fixing that by adding an extra check on the real number of replicas a block has before adding it to the list returned by BlockManager.getCorruptFiles() (previously BockManager.getBadFiles()). Please let me know what you guys think about it.
          Hide
          Rodrigo Schmidt added a comment -

          Something else I noticed on this API is that we are returning all block locations of the corrupt files, and not only the locations of those blocks that are corrupt. And besides block locations, we don't give any other information to the client about the file.

          I suggest that we return a map from FileStatus to corrupt block locations, listing all the files that are corrupt together with its corrupt blocks (leaving out the blocks that have at least one replica).

          Show
          Rodrigo Schmidt added a comment - Something else I noticed on this API is that we are returning all block locations of the corrupt files, and not only the locations of those blocks that are corrupt. And besides block locations, we don't give any other information to the client about the file. I suggest that we return a map from FileStatus to corrupt block locations, listing all the files that are corrupt together with its corrupt blocks (leaving out the blocks that have at least one replica).
          Hide
          Rodrigo Schmidt added a comment -

          Here is an initial proposal to the new API, returning a map from FileStatus to a LocatedBlocks object containing only the list of corrupted blocks.

          I'm not sure if this is the best interface and I'd like to hear your comments about it.

          Thanks!

          PS: I haven't put the unit tests yet because I wanted to get some consensus on the API first (on that respect, the unit tests would only make the patch look more complicated).

          Show
          Rodrigo Schmidt added a comment - Here is an initial proposal to the new API, returning a map from FileStatus to a LocatedBlocks object containing only the list of corrupted blocks. I'm not sure if this is the best interface and I'd like to hear your comments about it. Thanks! PS: I haven't put the unit tests yet because I wanted to get some consensus on the API first (on that respect, the unit tests would only make the patch look more complicated).
          Hide
          dhruba borthakur added a comment -

          One option would be to add ClientProtocol.getBadFiles

          public FileStatus[] getBadFiles()

          Also, please note that some changes to FileStatus are ocuring via HDFS-946

          Show
          dhruba borthakur added a comment - One option would be to add ClientProtocol.getBadFiles public FileStatus[] getBadFiles() Also, please note that some changes to FileStatus are ocuring via HDFS-946
          Hide
          Rodrigo Schmidt added a comment -

          I just added a complete patch.

          I'm returning an array of FileStatus objects, as Dhruba suggested, and I used the nomenclature proposed by Raghu (corrupt instead of bad)

          Show
          Rodrigo Schmidt added a comment - I just added a complete patch. I'm returning an array of FileStatus objects, as Dhruba suggested, and I used the nomenclature proposed by Raghu (corrupt instead of bad)
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12436392/HDFS-729.2.patch
          against trunk revision 911744.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12436392/HDFS-729.2.patch against trunk revision 911744. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/240/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          I went through the logs and there was no failed unit test. The build failed with the following error, which has nothing to do with this patch:

          BUILD FAILED
          [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build.xml:569: The following error occurred while executing this line:
          [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/src/contrib/build.xml:48: The following error occurred while executing this line:
          [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/src/contrib/hdfsproxy/build.xml:292: org.codehaus.cargo.container.ContainerException: Failed to download http://apache.osuosl.org/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip

          Show
          Rodrigo Schmidt added a comment - I went through the logs and there was no failed unit test. The build failed with the following error, which has nothing to do with this patch: BUILD FAILED [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build.xml:569: The following error occurred while executing this line: [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/src/contrib/build.xml:48: The following error occurred while executing this line: [exec] /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/src/contrib/hdfsproxy/build.xml:292: org.codehaus.cargo.container.ContainerException: Failed to download http://apache.osuosl.org/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip
          Hide
          dhruba borthakur added a comment -

          Code looks good. A few comments:

          1. TestDFSClientRetries.java, you do not need to import java.util.Map
          2. ClientProtocol javadocs still say "@return Map from FileStatus to LocatedBlocks object with locations of * corrupted blocks.".
          3. BlockManager.getCorruptInodes() returns a map of Inodes to BlockList. Instead, won't it be sufficient to return just a list of INodes?

          Show
          dhruba borthakur added a comment - Code looks good. A few comments: 1. TestDFSClientRetries.java, you do not need to import java.util.Map 2. ClientProtocol javadocs still say "@return Map from FileStatus to LocatedBlocks object with locations of * corrupted blocks.". 3. BlockManager.getCorruptInodes() returns a map of Inodes to BlockList. Instead, won't it be sufficient to return just a list of INodes?
          Hide
          dhruba borthakur added a comment -

          Oh, and maybe the unit test can check that no more than 500 files are being returned by this call (even if there are plenty more corrupted files). For this unit test, you might want to initiaize MAX_CORRUPT_FILES_RETURNED from a config parameter (default 500).

          Show
          dhruba borthakur added a comment - Oh, and maybe the unit test can check that no more than 500 files are being returned by this call (even if there are plenty more corrupted files). For this unit test, you might want to initiaize MAX_CORRUPT_FILES_RETURNED from a config parameter (default 500).
          Hide
          Rodrigo Schmidt added a comment -

          Thanks for the comments, Dhruba. 1 and 2 are ghosts from previous code. Good catches!

          3 is partly because it was the original code and partly because I thought that returning the blocks could be useful in future extensions. But I think you are right. Let's keep it simple and extend it in the future if needed.

          I'll make the changes you propose, create the new unit test and resubmit the patch.

          Show
          Rodrigo Schmidt added a comment - Thanks for the comments, Dhruba. 1 and 2 are ghosts from previous code. Good catches! 3 is partly because it was the original code and partly because I thought that returning the blocks could be useful in future extensions. But I think you are right. Let's keep it simple and extend it in the future if needed. I'll make the changes you propose, create the new unit test and resubmit the patch.
          Hide
          dhruba borthakur added a comment -

          Thanks Rodrigo. I will wait for your new patch.

          Show
          dhruba borthakur added a comment - Thanks Rodrigo. I will wait for your new patch.
          Hide
          Rodrigo Schmidt added a comment -

          new patch attached. (3 is my lucky number)

          I made all the changes suggested by Dhruba.

          Show
          Rodrigo Schmidt added a comment - new patch attached. (3 is my lucky number) I made all the changes suggested by Dhruba.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12437530/HDFS-729.3.patch
          against trunk revision 916902.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437530/HDFS-729.3.patch against trunk revision 916902. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/256/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          The errors are the same as before, and the same other patches seem to be going through.

          Dhruba, could you please double check that everything is fine with this patch?

          Show
          Rodrigo Schmidt added a comment - The errors are the same as before, and the same other patches seem to be going through. Dhruba, could you please double check that everything is fine with this patch?
          Hide
          dhruba borthakur added a comment -

          Code looks good. The only question I have is that BlockManager.getCorruptInodes does the following:

              LinkedHashSet<INode> set = 
                new LinkedHashSet<INode>(this.maxCorruptFilesReturned*2);
          
          

          Can you pl explain why the multiplication by 2 is needed?

          Show
          dhruba borthakur added a comment - Code looks good. The only question I have is that BlockManager.getCorruptInodes does the following: LinkedHashSet<INode> set = new LinkedHashSet<INode>( this .maxCorruptFilesReturned*2); Can you pl explain why the multiplication by 2 is needed?
          Hide
          Rodrigo Schmidt added a comment -

          Dhruba convinced me that starting with a big hash table is not a good idea. I'm uploading a new patch. Turns out 3 is not my lucky number any more.

          Show
          Rodrigo Schmidt added a comment - Dhruba convinced me that starting with a big hash table is not a good idea. I'm uploading a new patch. Turns out 3 is not my lucky number any more.
          Hide
          Rodrigo Schmidt added a comment -

          Almost the same thing as the previous patch, but with the default initial capacity for the LinkedHashSet object.

          Show
          Rodrigo Schmidt added a comment - Almost the same thing as the previous patch, but with the default initial capacity for the LinkedHashSet object.
          Hide
          Rodrigo Schmidt added a comment -

          Current patch file has some wrong comments and extra lines. I've just cleaned it up.

          Show
          Rodrigo Schmidt added a comment - Current patch file has some wrong comments and extra lines. I've just cleaned it up.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12437662/HDFS-729.4.patch
          against trunk revision 916902.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437662/HDFS-729.4.patch against trunk revision 916902. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/122/console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12437665/HDFS-729.4.patch
          against trunk revision 916902.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437665/HDFS-729.4.patch against trunk revision 916902. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/123/console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12437665/HDFS-729.4.patch
          against trunk revision 916902.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437665/HDFS-729.4.patch against trunk revision 916902. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/261/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          Forgot to change the version on ClientProtocol.java

          Show
          Rodrigo Schmidt added a comment - Forgot to change the version on ClientProtocol.java
          Hide
          dhruba borthakur added a comment -

          +1 Code looks good.

          Show
          dhruba borthakur added a comment - +1 Code looks good.
          Hide
          Rodrigo Schmidt added a comment -

          Apparently Hudson didn't pick up my last patch. I'll resubmit it.

          Show
          Rodrigo Schmidt added a comment - Apparently Hudson didn't pick up my last patch. I'll resubmit it.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12437729/HDFS-729.5.patch
          against trunk revision 916902.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12437729/HDFS-729.5.patch against trunk revision 916902. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/265/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          Hudson probably had some problem with the previous patch. It passes my unit tests.

          The new version was merged with the current trunk.

          Show
          Rodrigo Schmidt added a comment - Hudson probably had some problem with the previous patch. It passes my unit tests. The new version was merged with the current trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12438071/HDFS-729.6.patch
          against trunk revision 919680.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12438071/HDFS-729.6.patch against trunk revision 919680. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/128/console This message is automatically generated.
          Hide
          Rodrigo Schmidt added a comment -

          The failure in the contrib test was the same error attempting to download Tomcat I've seen in several other HDFS Jiras. It's completely unrelated to this patch:

          >>>>> Failed to download http://apache.osuosl.org/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip

          Show
          Rodrigo Schmidt added a comment - The failure in the contrib test was the same error attempting to download Tomcat I've seen in several other HDFS Jiras. It's completely unrelated to this patch: >>>>> Failed to download http://apache.osuosl.org/tomcat/tomcat-6/v6.0.18/bin/apache-tomcat-6.0.18.zip
          Hide
          dhruba borthakur added a comment -

          Thanks Rodrigo, I will commit this patch.

          Show
          dhruba borthakur added a comment - Thanks Rodrigo, I will commit this patch.
          Hide
          dhruba borthakur added a comment -

          I just committed this. Thanks Rodrigo!

          Show
          dhruba borthakur added a comment - I just committed this. Thanks Rodrigo!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/209/)
          . NameNode API to list files that have missing blocks.
          (Rodrigo Schmidt via dhruba)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #209 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/209/ ) . NameNode API to list files that have missing blocks. (Rodrigo Schmidt via dhruba)
          Hide
          Hudson added a comment -

          Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/)

          Show
          Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/ )
          Hide
          Hudson added a comment -

          Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #302 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/302/)

          Show
          Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #302 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/302/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/ )
          Hide
          Konstantin Shvachko added a comment -

          Could you please update components and version fields for this jira.

          Show
          Konstantin Shvachko added a comment - Could you please update components and version fields for this jira.

            People

            • Assignee:
              Rodrigo Schmidt
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development