Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1658

[UMBRELLA] Spark Sql Support For Hudi

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: In Progress
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: Spark Integration
    • Labels:

      Description

      This is the main task for supporting spark sql for hudi, including the DDL、DML and Hoodie CLI command.

        Attachments

          Issue Links

          1.
          Basic implementation Of Spark Sql Support Sub-task Resolved pengzhiwei
          2.
          Support SQL with spark3 Sub-task Closed tao meng
          3.
          Support Clustering and Metatable for SQL performance Sub-task Open Unassigned
          4.
          Support AlterCommand For Hoodie Sub-task Resolved pengzhiwei
          5.
          Support Cluster By In Create Table Sub-task Open Unassigned
          6.
          Support Create Index In Create Table Sub-task Open Unassigned
          7.
          Support Hoodie CLI Command In Spark SQL Sub-task Open Unassigned
          8.
          Add Commit Time Prefix To The UuidKeyGenerator For Insert Only Sub-task Open pengzhiwei
          9.
          [SQL] Spark Sql Support For The Exists Hoodie Table Sub-task Resolved pengzhiwei
          10.
          Support Spark 3.1(Duplicated) Sub-task Closed Unassigned
          11.
          Upgrading Spark3 To 3.1 Sub-task Open pengzhiwei
          12.
          Support Truncate Table For Hoodie Sub-task Resolved pengzhiwei
          13.
          MergeInto Support Partial Update For COW Sub-task Resolved pengzhiwei
          14.
          Support Delete/Update Non-Pk Table Sub-task Open Raymond Xu
          15.
          Performance testing/certification of key SQL DMLs Sub-task Open Raymond Xu
          16.
          Enable Hive Sync When Spark Enable Hive Meta For Spark Sql Sub-task Resolved pengzhiwei
          17.
          [SQL] Add Doc For Spark Sql Integrates With Hudi Sub-task Resolved pengzhiwei
          18.
          Support Alter table drop column Sub-task Open pengzhiwei
          19.
          Support Compaction Command For Spark Sql Sub-task Resolved pengzhiwei
          20.
          [SQL] Support Bulk Insert For Spark Sql Sub-task Resolved pengzhiwei
          21.
          Missing PrimaryKey In Hoodie Properties For CTAS Table Sub-task Resolved pengzhiwei
          22.
          [SQL] Functionality testing with Spark 2 Sub-task Closed Sagar Sumit
          23.
          [SQL] Test catalog integration Sub-task Closed Sagar Sumit
          24.
          [SQL] MERGE INTO fails with table having nested struct Sub-task Resolved pengzhiwei
          25.
          [SQL] Hive sync is not working Sub-task Resolved pengzhiwei
          26.
          MERGE INTO works only ON primary key Sub-task Open pengzhiwei
          27.
          [SQL] Changing index type fails Sub-task Resolved sivabalan narayanan
          28.
          [SQL] Bulk insert support for tables w/ primary key Sub-task Resolved pengzhiwei
          29.
          [SQL] Fix Exception Cause By Table Name Case Sensitivity For Append Mode Write Sub-task Resolved pengzhiwei
          30.
          Upgrade hoodie table to 0.9.0 Sub-task Resolved sivabalan narayanan
          31.
          Insert for an already existing record throws DuplicateKeyException with primary keyed spark sql table Sub-task Resolved pengzhiwei
          32.
          Support Clustering Command For Spark Sql Sub-task Open Unassigned
          33.
          Support delete partitions via alter table Sub-task Reopened Unassigned
          34.
          MERGE INTO doesn't work for tables created using CTAS Sub-task Closed pengzhiwei
          35.
          [SQL]Support referencing subquery with column aliases by table alias in merge into Sub-task Open 董可伦
          36.
          Fix the exception for mergeInto when the primaryKey and preCombineField of source table and target table differ in case only Sub-task Open 董可伦
          37.
          Introduce config to allow users to control case-sensitivity in column projections #431 Sub-task New Unassigned
          38.
          Support Multipath query for HoodieFileIndex Sub-task Open pengzhiwei
          39.
          Support drop partitions SQL Sub-task Open Yann Byron
          40.
          Support show partitions SQL Sub-task In Progress Yann Byron
          41.
          Delete data is not working with 0.9.0 and pySpark Sub-task Open Unassigned
          42.
          use commit_time in the WHERE STATEMENT to optimize the incremental query Sub-task Open David_Liang
          43.
          Create Table If Not Exists Failed After Alter Table Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                pzw2018 pengzhiwei
                Reporter:
                pzw2018 pengzhiwei
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: