Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7361

Main Jetty thread blocked by core loading delays HTTP listener from binding if core loading is slow

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 5.3, 6.0
    • SolrCloud
    • None

    Description

      During server startup, the CoreContainer uses an ExecutorService to load cores in multiple back-ground threads but then blocks until cores are loaded, see: CoreContainer#load around line 290 on trunk (invokeAll). From the JavaDoc on that method, we have:

      Executes the given tasks, returning a list of Futures holding their status and results when all complete. Future.isDone() is true for each element of the returned list.

      In other words, this is a blocking call.

      This delays the Jetty HTTP listener from binding and accepting requests until all cores are loaded. Do we need to block the main thread?

      Also, prior to this happening, the node is registered as a live node in ZK, which makes it a candidate for receiving requests from the Overseer, such as to service a create collection request. The problem of course is that the node listed in /live_nodes isn't accepting requests yet. So we either need to unblock the main thread during server loading or maybe wait longer before we register as a live node ... not sure which is the better way forward?

      Attachments

        1. SOLR-7361.patch
          33 kB
          Mark Miller
        2. SOLR-7361.patch
          32 kB
          Mark Miller
        3. SOLR-7361.patch
          32 kB
          Mark Miller
        4. SOLR-7361.patch
          15 kB
          Mark Miller
        5. SOLR-7361.patch
          14 kB
          Mark Miller
        6. SOLR-7361.patch
          14 kB
          Mark Miller
        7. SOLR-7361.patch
          9 kB
          Mark Miller
        8. SOLR-7361.patch
          8 kB
          Timothy Potter

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            markrmiller@gmail.com Mark Miller
            thelabdude Timothy Potter
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment