Uploaded image for project: 'ManifoldCF'
  1. ManifoldCF
  2. CONNECTORS-1249

Keep a separate document priority queue per job, and synchronize with any running jobs on job start

    XMLWordPrintableJSON

    Details

      Description

      Starting a job when there has been already a long-running job in MCF takes a very long time, because the documents from the new job don't get processed until the other jobs' current backlog at the time the new job was started go away.

      Effectively, this is because there is only one stream of document priorities, and all jobs tap into that. But there's no reason why we can't have multiple document priority streams, one per active job, with some redesign work.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                kwright@metacarta.com Karl Wright
                Reporter:
                kwright@metacarta.com Karl Wright
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: