Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14118

Implement DDL/DML commands for Spark 2.0

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 2.0.0
    • SQL
    • None

    Description

      Right now, we have many DDL/DML commands that are passed to Hive, which may cause missing functionality, failures with bad error messages, or inconsistent behaviors (e.g. a command that works with some cases but fails for other cases). For Spark 2.0, it will be great to not ask Hive to process those DDL/DML commands.

      You can find the doc at https://issues.apache.org/jira/secure/attachment/12793435/Implementing%20native%20DDL%20and%20DML%20statements%20for%20Spark%202.pdf (under SPARK-13879).

      There are mainly two kinds of commands, (1) Native, i.e. we want to have native implementation in Spark; (2) Exception, i.e. we should throw an exception. That doc has a few commands that are marked as TBD. We should first throw exceptions for them.

      Sub-tasks are created based on the doc. A command is represented by its corresponding Token.

      Attachments

        Issue Links

        There are no Sub-Tasks for this issue.

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            yhuai Yin Huai Assign to me
            yhuai Yin Huai
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment