Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18799

Spark SQL expose interface for plug-gable parser extension

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.0.0
    • None
    • SQL
    • None
    • Important

    Description

      There used to be an interface to plug a parser extension through ParserDialect in HiveContext in all Spark 1.x version. Starting Spark 2.x release, Apache Spark moved to the new parser (Antlr4), there is no longer a way to extend the default SQL parser through SparkSession interface, however this is really a pain and hard to work around it when integrating other data source with Spark with extended support such as Insert, Update, Delete statement or any other data management statement.

      It would be very nice to continue to expose an interface for parser extension to make data source integration easier and smoother.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            jihongma Jihong Ma
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment