[AURORA-139] Scheduling performance could be vastly improved by caching job information

• Type: Story
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.0
• Component/s: Scheduler
• Assignee: Bill Farner
• Reporter: Bill Farner
• Created: 29/Jan/14 18:33
• Updated: 27/Mar/14 17:20
• Resolved: 31/Jan/14 07:35

Description

SchedulingFilterImpl has this snippet of code:

            Supplier<Collection<IScheduledTask>> activeTasksSupplier =
                Suppliers.memoize(new Supplier<Collection<IScheduledTask>>() {
                  @Override public Collection<IScheduledTask> get() {
                    return storeProvider.getTaskStore().fetchTasks(
                        Query.jobScoped(Tasks.INFO_TO_JOB_KEY.apply(task))
                            .byStatus(ACTIVE_NOT_PENDING_STATES));
                  }
                });

This results in a job-scoped query for every offer being compared to a task. Since this operation is already in the scope of a write lock, we could precompute this information from outside the loop over offers and do the expensive query only once.

Issue Links

• relates to: AURORA-117 Scheduler performance issues with very large jobs