Details
Description
One possible feature would be to add a new endpoint for indexing-backends and make the indexing plugable. at the moment we are hardwired to SOLR - which is OK - but as other resources like ElasticSearch are becoming more popular it would be better to handle this as plugins. Not sure about the name of the endpoint though : we already have indexing-plugins (which are about generating fields sent to the backends) and moreover the backends are not necessarily for indexing / searching but could be just an external storage e.g. CouchDB. The term backend on its own would be confusing in 2.0 as this could be pertaining to the storage in GORA. 'indexing-backend' is the best name that came to my mind so far - please suggest better ones.
We should come up with generic map/reduce jobs for indexing, deduplicating and cleaning and maybe add a Nutch extension point there so we can easily hook up indexing, cleaning and deduplicating for various backends.
Attachments
Attachments
Issue Links
- blocks
-
NUTCH-1527 Port nutch-elasticsearch-indexer to Nutch
-
- Closed
-
-
NUTCH-1528 Port nutch-mongodb-indexer to Nutch
-
- Closed
-
- is depended upon by
-
NUTCH-1088 Write Solr XML documents
-
- Open
-
-
NUTCH-1517 CloudSearch indexer
-
- Closed
-
- is related to
-
NUTCH-1446 Port NUTCH-1444 to trunk (Indexing should not create temporary files)
-
- Open
-
-
NUTCH-1139 Indexer to delete documents
-
- Closed
-
-
NUTCH-656 DeleteDuplicates based on crawlDB only
-
- Closed
-
- relates to
-
NUTCH-1568 port pluggable indexing architecture to 2.x
-
- Closed
-