[#LUCENE-1084] increase default maxFieldLength?

Lucene - Java

increase default maxFieldLength?

Created: 07/Dec/07 10:36 PM Updated: 11/Oct/08 12:49 PM

Return to search

Component/s:

Index

Affects Version/s:

2.2

Fix Version/s:

2.4

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
LUCENE-1084.part2.patch	2008-02-07 04:14 PM	Steven Rowe	2 kB
LUCENE-1084.part2.take2.patch	2008-02-07 04:52 PM	Steven Rowe	7 kB
LUCENE-1084.part3.patch	2008-02-07 06:49 PM	Steven Rowe	4 kB
LUCENE-1084.patch	2008-01-17 06:35 PM	Steven Rowe	64 kB

Lucene Fields:	Patch Available, New
Resolution Date:	07/Feb/08 06:22 PM

Description

« Hide

To my understanding, Lucene 2.3 will easily index large documents. So shouldn't we get rid of the 10,000 default limit for the field length? 10,000 isn't that much and as Lucene doesn't have any error logging by default, this is a common problem for users that is difficult to debug if you don't know where to look.

A better new default might be Integer.MAX_VALUE.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#615168	Fri Jan 25 10:21:15 UTC 2008	mikemccand	~~LUCENE-1084~~: change all IW ctors to require up-front specification of max field length; deprecated existing ones
				Files Changed
				MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestFilterIndexReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestPositionIncrement.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestRemoteCachingWrapperFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/BaseTestRangeFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestTermVectors.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/spans/TestBasics.java MODIFY /lucene/java/trunk/contrib/wordnet/src/java/org/apache/lucene/wordnet/Syns2Index.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/store/TestLockFactory.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestSimilarity.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestSpanQueryFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestSearchForDuplicates.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestSnapshotDeletionPolicy.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestPhraseQuery.java MODIFY /lucene/java/trunk/CHANGES.txt MODIFY /lucene/java/trunk/contrib/benchmark/src/java/org/apache/lucene/benchmark/standard/StandardBenchmarker.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestExplanations.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestDemo.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestHitIterator.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestSegmentTermDocs.java MODIFY /lucene/java/trunk/contrib/surround/src/test/org/apache/lucene/queryParser/surround/query/SingleFieldTestDb.java MODIFY /lucene/java/trunk/contrib/miscellaneous/src/java/org/apache/lucene/misc/IndexMergeTool.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestDateSort.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/IndexTest.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/function/FunctionTestSetup.java MODIFY /lucene/java/trunk/src/demo/org/apache/lucene/demo/IndexFiles.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcherRanking.java MODIFY /lucene/java/trunk/contrib/regex/src/test/org/apache/lucene/search/regex/TestRegexQuery.java MODIFY /lucene/java/trunk/contrib/swing/src/java/org/apache/lucene/swing/models/ListSearcher.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestConstantScoreRangeQuery.java MODIFY /lucene/java/trunk/src/demo/org/apache/lucene/demo/IndexHTML.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestSearch.java MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/IndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/analysis/TestCachingTokenFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestCheckIndex.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestMultiLevelSkipList.java

Repository	Revision	Date	User	Message
ASF	#617104	Thu Jan 31 11:23:38 UTC 2008	mikemccand	~~LUCENE-1084~~: fixing a few more IndexWriter instantiations to not use deprecated ctors
				Files Changed
				MODIFY /lucene/java/trunk/src/test/org/apache/lucene/ThreadSafetyTest.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/SearchTest.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriter.java

Repository	Revision	Date	User	Message
ASF	#618607	Tue Feb 05 11:12:53 UTC 2008	mikemccand	~~LUCENE-1084~~: fix some more deprecated IndexWriter instantiations in unit tests
				Files Changed
				MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestMergeSchedulerExternal.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/analysis/TestKeywordAnalyzer.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/SearchTestForDuplicates.java

Repository	Revision	Date	User	Message
ASF	#618764	Tue Feb 05 20:07:29 UTC 2008	mikemccand	~~LUCENE-1084~~: fix the remaining deprecated calls to IndexWriter ctors
				Files Changed
				MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiPhraseQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestDeletionPolicy.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestDisjunctionMaxQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestAddIndexesNoOptimize.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestDoc.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestBooleanPrefixQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestTermdocPerf.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestSetNorm.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestBoolean2.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/document/TestDocument.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestLazyProxSkipping.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiThreadTermVectors.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestNot.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestMatchAllDocsQuery.java MODIFY /lucene/java/trunk/contrib/benchmark/src/test/org/apache/lucene/benchmark/byTask/TestPerfTasksLogic.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestMultiSearcher.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestPayloads.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestBooleanMinShouldMatch.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestBooleanScorer.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/spans/TestNearSpansOrdered.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/spans/TestSpansAdvanced.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestMultiSegmentReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/spans/TestSpans.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/spans/TestSpansAdvanced2.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestTermScorer.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestRangeQuery.java MODIFY /lucene/java/trunk/contrib/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/CreateIndexTask.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestCustomSearcherSort.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestExtendedFieldCache.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestTermVectorsReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestFuzzyQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestParallelReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestFilteredQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/store/TestBufferedIndexInput.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestWildcard.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestThreadedOptimize.java MODIFY /lucene/java/trunk/contrib/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/OpenIndexTask.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestBooleanOr.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/queryParser/TestQueryParser.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestBackwardsCompatibility.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestFieldsReader.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/DocHelper.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriterLockRelease.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestPhrasePrefixQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestScorerPerf.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/document/TestBinaryDocument.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestAtomicUpdate.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriterDelete.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestDateFilter.java MODIFY /lucene/java/trunk/contrib/ant/src/java/org/apache/lucene/ant/IndexTask.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriterMerging.java MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/IndexModifier.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestSearchHitsWithDeletions.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriterMergePolicy.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestSegmentTermEnum.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestLazyBug.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/queryParser/TestMultiFieldQueryParser.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/store/TestRAMDirectory.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestNorms.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/payloads/TestBoostingTermQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestConcurrentMergeScheduler.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexFileDeleter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestCachingWrapperFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestDocBoost.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestPrefixFilter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReaderReopen.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestRemoteSearchable.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestParallelTermEnum.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestPrefixQuery.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestDocumentWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestSort.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/store/TestWindowsMMap.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestStressIndexing.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/search/TestThreadSafe.java

Repository	Revision	Date	User	Message
ASF	#619545	Thu Feb 07 18:22:21 UTC 2008	mikemccand	~~LUCENE-1084~~: allow user-specified max field length when instantiating IndexWriter
				Files Changed
				MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/IndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriter.java

Repository	Revision	Date	User	Message
ASF	#619637	Thu Feb 07 21:07:35 UTC 2008	mikemccand	~~LUCENE-1084~~: fix more test cases to use user-defined MaxFieldLength ctor for IndexWriter
				Files Changed
				MODIFY /lucene/java/trunk/src/demo/org/apache/lucene/demo/IndexHTML.java MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/IndexModifier.java MODIFY /lucene/java/trunk/contrib/memory/src/test/org/apache/lucene/index/memory/MemoryIndexTest.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/TestDemo.java