[SOLR-5658] commitWithin does not reflect the new documents added

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.6, 6.0
• Fix Version/s: 4.6.1, 4.7, 6.0
• Assignee: Mark Miller
• Reporter: Varun Thacker
• Created: 23/Jan/14 10:38
• Updated: 09/May/16 18:49
• Resolved: 22/Feb/14 20:40

Description

I start 4 nodes using the setup mentioned on - https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud

I added a document using -
curl http://localhost:8983/solr/update?commitWithin=10000 -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">testdoc</field></doc></add>'

In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit with openSearcher=false
In Solr 4.6.x there is there is only one commit hard commit with openSearcher=false

So even after 10 seconds queries on none of the shards reflect the added document.

This was also reported on the solr-user list ( http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html )

Here are the relevant logs

Logs from Solr 4.5.1
Node 1:

420021 [qtp619011445-12] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 45

Node 2:

119896 [qtp1608701025-10] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003295513608192)]} 0 348
129648 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
129679 [commitScheduler-8-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@e174f70 main
129680 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener sending requests to Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener done.
129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – [collection1] Registered new searcher Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
134648 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
134658 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
134658 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 4
134660 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
 

Node 3:

Node 4:

374545 [qtp1608701025-16] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458002133233172480)]} 0 20
384545 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
384552 [commitScheduler-8-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@36137e08 main
384553 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
384553 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener sending requests to Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
384553 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener done.
384554 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – [collection1] Registered new searcher Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
389545 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
389549 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
389550 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
389551 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush

Using Solr 4.6

Node 1:

124513 [qtp1314570047-13] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 348

Node 2:

101586 [qtp1608701025-13] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 217
116407 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
116429 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
116430 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 2
116444 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@75e32318 realtime
116445 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
 

Node 3:

Node 4:

68183 [qtp1338008566-14] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 43
83183 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
83207 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
83208 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 2
83220 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@326f944c realtime
83220 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
 

Using Solr 4.6.1

Node 1:

301363 [qtp619011445-15] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 32

Node 2:

207000 [qtp619011445-17] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 28
221974 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
221987 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
221987 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
221989 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@132713fa realtime
221990 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush

Node 3:

Node 4:

193133 [qtp1608701025-16] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 23
208133 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
208141 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
208141 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
208144 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@3171c7df realtime
208146 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush

Issue Links

• breaks: SOLR-5762 SOLR-5658 broke backward compatibility of Javabin format