Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-5658

commitWithin does not reflect the new documents added

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 4.6, 6.0
    • 4.6.1, 4.7, 6.0
    • None
    • None

    Description

      I start 4 nodes using the setup mentioned on - https://cwiki.apache.org/confluence/display/solr/Getting+Started+with+SolrCloud

      I added a document using -
      curl http://localhost:8983/solr/update?commitWithin=10000 -H "Content-Type: text/xml" --data-binary '<add><doc><field name="id">testdoc</field></doc></add>'

      In Solr 4.5.1 there is 1 soft commit with openSearcher=true and 1 hard commit with openSearcher=false
      In Solr 4.6.x there is there is only one commit hard commit with openSearcher=false

      So even after 10 seconds queries on none of the shards reflect the added document.

      This was also reported on the solr-user list ( http://lucene.472066.n3.nabble.com/Possible-regression-for-Solr-4-6-0-commitWithin-does-not-work-with-replicas-td4106102.html )

      Here are the relevant logs

      Logs from Solr 4.5.1
      Node 1:

      420021 [qtp619011445-12] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 45
      

      Node 2:

      119896 [qtp1608701025-10] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003295513608192)]} 0 348
      129648 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
      129679 [commitScheduler-8-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@e174f70 main
      129680 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
      129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener sending requests to Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
      129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener done.
      129681 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – [collection1] Registered new searcher Searcher@e174f70 main{StandardDirectoryReader(segments_3:11:nrt _2(4.5.1):C1)}
      134648 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      134658 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66a394a3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
      134658 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 4
      134660 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
       

      Node 3:

      Node 4:

      374545 [qtp1608701025-16] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458002133233172480)]} 0 20
      384545 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
      384552 [commitScheduler-8-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@36137e08 main
      384553 [commitScheduler-8-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
      384553 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener sending requests to Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
      384553 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – QuerySenderListener done.
      384554 [searcherExecutor-5-thread-1] INFO  org.apache.solr.core.SolrCore  – [collection1] Registered new searcher Searcher@36137e08 main{StandardDirectoryReader(segments_2:7:nrt _1(4.5.1):C1)}
      389545 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      389549 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.5.1/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4d4c84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
      389550 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
      389551 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
      

      Using Solr 4.6

      Node 1:

      124513 [qtp1314570047-13] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 348
      

      Node 2:

      101586 [qtp1608701025-13] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 217
      116407 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      116429 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@245e7588; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
      116430 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 2
      116444 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@75e32318 realtime
      116445 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
       

      Node 3:

      Node 4:

      68183 [qtp1338008566-14] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458003613357965312)]} 0 43
      83183 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      83207 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/solr-4.6.0/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@69c9fc69; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
      83208 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 2
      83220 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@326f944c realtime
      83220 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
       

      Using Solr 4.6.1

      Node 1:

      301363 [qtp619011445-15] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={commitWithin=10000} {add=[testdoc]} 0 32
      

      Node 2:

      207000 [qtp619011445-17] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:8983/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 28
      221974 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      221987 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node2/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@352b9aeb; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
      221987 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
      221989 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@132713fa realtime
      221990 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
      

      Node 3:

      Node 4:

      193133 [qtp1608701025-16] INFO  org.apache.solr.update.processor.LogUpdateProcessor  – [collection1] webapp=/solr path=/update params={distrib.from=http://192.168.1.103:7574/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[testdoc (1458004563169640448)]} 0 23
      208133 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
      208141 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – SolrDeletionPolicy.onCommit: commits: num=2
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
      	commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/varun/Downloads/search-downloads/solr-4.6.1/solr/node4/solr/collection1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3f83dcf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
      208141 [commitScheduler-7-thread-1] INFO  org.apache.solr.core.SolrCore  – newest commit generation = 3
      208144 [commitScheduler-7-thread-1] INFO  org.apache.solr.search.SolrIndexSearcher  – Opening Searcher@3171c7df realtime
      208146 [commitScheduler-7-thread-1] INFO  org.apache.solr.update.UpdateHandler  – end_commit_flush
      

      Attachments

        1. SOLR-5658.patch
          14 kB
          Mark Miller
        2. SOLR-5658.patch
          3 kB
          Mark Miller

        Issue Links

          Activity

            People

              markrmiller@gmail.com Mark Miller
              varun Varun Thacker
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: