The RaidNode is not currently using this API, although its use was one of the motivations I had for adding getCorruptFiles() to ClientProtocol. Originally, raid was part of HDFS and I could certainly see how Raid (and possibly other parts of HDFS) could benefit from this as an RPC to the namenode. I thought the others saw it too because when I got to
HDFS-729, having getCorruptFiles() on ClientProtocol was not under discussion anymore.
The JIRA that is responsible for making the RaidNode call getCorruptFiles is
HDFS-1171. Most probably we will have to extend DistributedFileSystem to export getCorruptFiles(). That's why I said we don't have to be external to HDFS, but we can be external to the namenode.
On the other hand, if we do take getCorruptFiles() out of ClientProtocol, we will make
HDFS-1171 overly complicated or expensive.
I really think the correct design choice is to export basic APIs like getCorruptFiles() as RPCs and build services like fsck and raid completely outside the namenode. After looking at the fsck code from the inside out and having experienced how it can sometimes compromise the whole filesystem because the namenode is using most of its resources to calculate outputs for fsck requests, I'm convinced it should be outside the namenode. For the sake of horizontal scalability of the namenode, we should be working in redesigning things like the current fsck implementation, instead of reinforcing it.
That's what I meant when I said we should be taking things out of the namenode. In my opinion, even if my case about having other parts of HDFS call getCorruptFiles() is not convincing, taking it out of ClientProtocol only reinforces the design choice of running fsck inside the namenode, which I think is bad. As we have more and more discussions about a distributed namenode, things like fsck should be the first ones running externally to it (to the namenode, not to HDFS). I see this as a low-hanging fruit towards a more scalable and distributed namenode.