Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-799

SOLRIndexer to commit once all reducers have finished

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.1
    • indexer
    • None
    • Patch Available

    Description

      What about doing only one SOLR commit after the MR job has finished in SOLRIndexer instead of doing that at the end of every Reducer?
      I ran into timeout exceptions in some of my reducers and I suspect that this was due to the fact that other reducers had already finished and called commit.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jnioche Julien Nioche
            jnioche Julien Nioche
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment