[#NUTCH-442] Integrate Solr/Nutch

Nutch

Integrate Solr/Nutch

Created: 07/Feb/07 06:37 PM Updated: 10/Apr/09 12:29 PM

Return to search

Component/s:

indexer, searcher

Affects Version/s:

None

Fix Version/s:

1.0.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
Crawl.patch	2008-05-12 08:19 PM	Caspar MacRae	3 kB
Indexer.patch	2008-05-14 02:04 AM	Caspar MacRae	15 kB
NUTCH-442_v4.patch	2007-11-19 09:12 PM	Doğacan Güney	182 kB
NUTCH-442_v5.patch	2008-04-17 02:14 PM	Doğacan Güney	178 kB
NUTCH-442_v6.patch.txt	2008-06-22 03:24 PM	Julien Nioche	181 kB
NUTCH-442_v7.patch.txt	2008-08-05 07:57 AM	Guillaume Smet	188 kB
NUTCH-442_v7a.patch.txt	2008-09-19 03:53 PM	Nick Tkach	183 kB
NUTCH-442_v8.patch	2008-10-09 12:00 PM	Doğacan Güney	192 kB
NUTCH_442_v3.patch	2007-08-06 08:47 AM	Doğacan Güney	196 kB
RFC_multiple_search_backends.patch	2007-07-31 01:19 PM	Doğacan Güney	158 kB
schema.xml	2007-07-31 02:01 PM	Doğacan Güney	2 kB

Environment:

Ubuntu linux

Resolution Date:

12/Jan/09 01:27 PM

Description

« Hide

Hi:

After trying out Sami's patch regarding Solr/Nutch. Can be found here (http://blog.foofactory.fi/2007/02/online-indexing-integrating-nutch-with.html) and I can confirm it worked And that lead me to request the following :

I would be very very great full if this could be included in nutch 0.9 as I am trying to eliminate my python based crawler which post documents to solr. As I am in the corporate enviornment I can't install trunk version in the production enviornment thus I am asking this to be included in 0.9 release. I hope my wish would be granted.

I look forward to get some feedback.

Thank you.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#733738	Mon Jan 12 13:26:16 UTC 2009	dogacan	~~NUTCH-442~~ - Integrate Solr/Nutch
				Files Changed
				MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/NutchBean.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneSearchBean.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SolrSearchBean.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrWriter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/FetchedSegments.java MODIFY /lucene/nutch/trunk/src/plugin/response-xml/src/java/org/apache/nutch/searcher/response/xml/XMLResponseWriter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/net/URLFilters.java ADD /lucene/nutch/trunk/lib/apache-solr-common-1.3.0.jar MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/IndexingFilter.java MODIFY /lucene/nutch/trunk/src/web/jsp/anchors.jsp ADD /lucene/nutch/trunk/lib/apache-solr-solrj-1.3.0.jar ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrConstants.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/IndexingFilters.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/scoring/ScoringFilters.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Summary.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/RPCSegmentBean.java MODIFY /lucene/nutch/trunk/src/web/jsp/cached.jsp MODIFY /lucene/nutch/trunk/build.xml MODIFY /lucene/nutch/trunk/src/plugin/feed/src/test/org/apache/nutch/parse/feed/TestFeedParser.java MODIFY /lucene/nutch/trunk/src/plugin/tld/src/java/org/apache/nutch/scoring/tld/TLDScoringFilter.java MODIFY /lucene/nutch/trunk/src/plugin/feed/src/java/org/apache/nutch/parse/feed/FeedParser.java MODIFY /lucene/nutch/trunk/src/web/jsp/search.jsp ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/RPCSearchBean.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/Indexer.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearchBean.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/crawl/Inlinks.java MODIFY /lucene/nutch/trunk/src/web/jsp/explain.jsp MODIFY /lucene/nutch/trunk/CHANGES.txt ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/IndexerOutputFormat.java MODIFY /lucene/nutch/trunk/bin/nutch MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryException.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/solr ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/lucene ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NutchDocument.java MODIFY /lucene/nutch/trunk/src/plugin/build.xml MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Hits.java MODIFY /lucene/nutch/trunk/src/plugin/index-more/src/java/org/apache/nutch/indexer/more/MoreIndexingFilter.java MODIFY /lucene/nutch/trunk/src/test/org/apache/nutch/indexer/TestIndexingFilters.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/lucene/LuceneWriter.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SearchBean.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NutchIndexWriter.java MODIFY /lucene/nutch/trunk/src/plugin/subcollection/src/java/org/apache/nutch/indexer/subcollection/SubcollectionIndexingFilter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/QueryFilters.java MODIFY /lucene/nutch/trunk/src/plugin/microformats-reltag/src/java/org/apache/nutch/microformats/reltag/RelTagIndexingFilter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/LuceneQueryOptimizer.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/OpenSearchServlet.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Hit.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/crawl/Crawl.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/lucene/LuceneConstants.java MODIFY /lucene/nutch/trunk/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang/NGramProfile.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/IndexSearcher.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/servlet/Cached.java MODIFY /lucene/nutch/trunk/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang/LanguageIdentifier.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSegmentBean.java MODIFY /lucene/nutch/trunk/src/plugin/index-anchor/src/java/org/apache/nutch/indexer/anchor/AnchorIndexingFilter.java MODIFY /lucene/nutch/trunk/src/plugin/tld/src/java/org/apache/nutch/indexer/tld/TLDIndexingFilter.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/SegmentBean.java MODIFY /lucene/nutch/trunk/src/plugin/scoring-link/src/java/org/apache/nutch/scoring/link/LinkAnalysisScoringFilter.java MODIFY /lucene/nutch/trunk/src/plugin/response-json/src/java/org/apache/nutch/searcher/response/json/JSONResponseWriter.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrIndexer.java MODIFY /lucene/nutch/trunk/src/plugin/scoring-opic/src/java/org/apache/nutch/scoring/opic/OPICScoringFilter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/HitDetails.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NutchIndexWriterFactory.java MODIFY /lucene/nutch/trunk/src/plugin/feed/src/java/org/apache/nutch/indexer/feed/FeedIndexingFilter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/scoring/ScoringFilter.java MODIFY /lucene/nutch/trunk/src/plugin/index-basic/src/java/org/apache/nutch/indexer/basic/BasicIndexingFilter.java ADD /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/IndexerMapReduce.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/DistributedSearch.java MODIFY /lucene/nutch/trunk/src/plugin/languageidentifier/src/test/org/apache/nutch/analysis/lang/TestNGramProfile.java MODIFY /lucene/nutch/trunk/src/plugin/creativecommons/src/java/org/creativecommons/nutch/CCIndexingFilter.java MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/searcher/Query.java MODIFY /lucene/nutch/trunk/src/plugin/languageidentifier/src/test/org/apache/nutch/analysis/lang/TestLanguageIdentifier.java MODIFY /lucene/nutch/trunk/src/plugin/languageidentifier/src/java/org/apache/nutch/analysis/lang/LanguageIndexingFilter.java

Repository	Revision	Date	User	Message
ASF	#733744	Mon Jan 12 13:30:28 UTC 2009	dogacan	Unrelated change went in accidentally in ~~NUTCH-442~~. Reverting to old version.
				Files Changed
				MODIFY /lucene/nutch/trunk/src/plugin/build.xml

Repository	Revision	Date	User	Message
ASF	#733848	Mon Jan 12 17:33:16 UTC 2009	dogacan	Two more ~~NUTCH-442~~ changes: * Delete TestDistributedSearch for now * Set reduceSpeculativeExecution false for SolrIndexer
				Files Changed
				MODIFY /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/solr/SolrIndexer.java DEL /lucene/nutch/trunk/src/test/org/apache/nutch/searcher/TestDistributedSearch.java