Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25232

Support Full-Text Search in Spark SQL

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.3.1
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:

      Description

      Full-text search (i.e., keyword search) is widely used in search engines and relational databases such as MATCH() ... AGAINST operator in MySQL (https://dev.mysql.com/doc/en/fulltext-search.html), Text query in Oracle (https://docs.oracle.com/cd/B28359_01/text.111/b28303/query.htm#g1016054), and text search in PostgreSQL (https://www.postgresql.org/docs/9.5/static/textsearch.html). However, it is not natively supported in Spark SQL. We propose an approach to implement this full-text search in Spark SQL.

      Our proposed approach is detailed  at https://github.com/JerryLead/Misc/blob/master/FullTextSearch/Full-text-issue-2018.pdf

      and the prototype is available at https://github.com/bigdata-iscas/SparkFullTextQuery/tree/like_explorer

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jerrylead Lijie Xu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: