Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-14952

Better search on the website

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: website
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      We use Algolia for the search functionality on the website using their free plan for Open Source projects. The index is built by Algolia's crawler using the DocSearch.

      When this was done we built our own UI on top of Algolia JavaScript API, as one if requirements is that clients use Algolia's JavaScript clients. We did not use Algolia UI as at that point it was rather large JavaScript dependency to add and it would slow down the loading of the website.

      We also have some initial work on creating our own Algolia index at build time.

      The current search doesn't seem to index the whole website, some results don't appear in the search, looks like most of the content from Antora is not indexed: trying to search for removeHeader, the FAQ entry is not found. There's also a list of failed searches on the Algolia dashboard we can use to benchmark the search.

      What we need is to build the search index over the whole content. Approach taken in #74 is good start for Hugo generated content. We need to expand that to Antora built content as well.

      This search index would be built at the website build time and would include both Hugo and Algolia content in the same file or possibly in several files if we use multi-index search. More on how indexes are built can be seen in the Algolia documentation.

      We need to figure out what data to send and how to integrate this with Antora, for Hugo we have a good idea from #74, importantly the structure needs to be the same. One good source of inspiration on building the index for Antora content is in the Lunr.js integration.

      We need to build the index with the search UI in mind, i.e. the index needs to contain the data we wish to present in the UI as well as enough content for Algolia to be able to use the content to perform search. So starting with a mockup of what we wish to present/utilize in the search UI and deriving the data structure for the index from that would be a good start.

        Attachments

        1. sitemap-camel.png
          104 kB
          Aemie
        2. image-2020-06-13-14-39-08-776.png
          28 kB
          Aashna Jena
        3. BH4D9OD16A_apache_camel_20200608-20200614_no_result_searches.csv
          6 kB
          Zoran Regvart
        4. List_Of_Crawled_Pages_by_DocSearch.txt
          130 kB
          Aashna Jena
        5. getbootstrap-searchresult.png
          60 kB
          Aemie
        6. error-search.png
          21 kB
          Aemie
        7. Screenshot_2020-07-08 Home - Apache Camel.png
          37 kB
          Zoran Regvart
        8. search-result-hazelcast.png
          69 kB
          Aemie
        9. apache_demo.json
          2 kB
          Aemie
        10. properties-new-config.png
          14 kB
          Aemie
        11. camel-test.json
          2 kB
          Aashna Jena

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                zregvart Zoran Regvart
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: