Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10317

Solr Nightly Benchmarks



    • Task
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None


      The benchmarking suite is now here: https://github.com/thesearchstack/solr-bench

      Actual datasets and queries are TBD yet.


      — Original description —
      Solr needs nightly benchmarks reporting. Similar Lucene benchmarks can be found here, https://home.apache.org/~mikemccand/lucenebench/.

      Preferably, we need:

      1. A suite of benchmarks that build Solr from a commit point, start Solr nodes, both in SolrCloud and standalone mode, and record timing information of various operations like indexing, querying, faceting, grouping, replication etc.
      2. It should be possible to run them either as an independent suite or as a Jenkins job, and we should be able to report timings as graphs (Jenkins has some charting plugins).
      3. The code should eventually be integrated in the Solr codebase, so that it never goes out of date.

      There is some prior work / discussion:

      1. https://github.com/shalinmangar/solr-perf-tools (Shalin)
      2. https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md (Ishan/Vivek)
      3. SOLR-2646 & SOLR-9863 (Mark Miller)
      4. https://home.apache.org/~mikemccand/lucenebench/ (Mike McCandless)
      5. https://github.com/lucidworks/solr-scale-tk (Tim Potter)

      There is support for building, starting, indexing/querying and stopping Solr in some of these frameworks above. However, the benchmarks run are very limited. Any of these can be a starting point, or a new framework can as well be used. The motivation is to be able to cover every functionality of Solr with a corresponding benchmark that is run every night.

      Proposing this as a GSoC 2017 project. I'm willing to mentor, and I'm sure shalinmangar and ~[markrmiller@gmail.com would help here.


        1. changes-lucene-20160907.json
          1.49 MB
          Alexandre Rafalovitch
        2. changes-solr-20160907.json
          1.71 MB
          Alexandre Rafalovitch
        3. managed-schema
          6 kB
          Alexandre Rafalovitch
        4. Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks.docx
          9 kB
          Vivek Narang
        5. Narang-Vivek-SOLR-10317-Solr-Nightly-Benchmarks-FINAL-PROPOSAL.pdf
          209 kB
          Vivek Narang
        6. Screenshot from 2017-07-30 20-30-05.png
          207 kB
          Ishan Chattopadhyaya
        7. SOLR-10317.patch
          524 kB
          Michael Sun
        8. SOLR-10317.patch
          36 kB
          Michael Sun
        9. solrconfig.xml
          0.9 kB
          Alexandre Rafalovitch

        Issue Links



              ichattopadhyaya Ishan Chattopadhyaya
              ichattopadhyaya Ishan Chattopadhyaya
              4 Vote for this issue
              19 Start watching this issue