Indexers currently rely on the LinkDB for anchor indexing while the WebGraph provides the same data as an inverted link DB. An inlinkDB created by the WebGraph program with non-zero LinkRank scores on the nodes also provide an improved set ordered by popularity.
This issue must:
- let IndexerMapReduce understand the new format;
- allow for indexing only popular anchors.
The goal is todeprecate all code associated with invertlinks and ultimately remove it from the codebase.