Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34123

Faster way to display/render entries in HistoryPage (Spark history server summary page)



    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: Web UI
    • Labels:


      Since a long time ago my team/company suffered from history server being very slow to display/search entries specially when entries grow over 50k entry, regardless there is a pagination there in that page already but still very slow to display the entries.


      Current situation Mustache Js is used to render the entries and datatables is used to manipulate it (sort by column and search).


      By getting rid of Mustache  (stop rendering the entries using Mustache) and using datatables  to display it proved to be faster.


      Displaying > 100k entries (my case):

      Existing takes at least 30 to 40 seconds to display the entries, searching takes at least 20 seconds and the page stop responding until it finishes.

      Improved takes ~3 seconds to display the entries searching is very fast and the page stays responsive.

      (These numbers will be different for others since JS is executed on your browser)


      I am not sure why Mustache is used to display the data since data tables can do the job,

      Alex Bozarth Sean Owen  please elaborate more about this what is the reason to use Mustache? what are the drawbacks if it's not used anymore to display the entries (only this part)?




            • Assignee:
              mohanad Mohanad Elsafty
              mohanad Mohanad Elsafty
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: