Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-1658

[UMBRELLA] Spark Sql Support For Hudi

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Blocker
    • Resolution: Unresolved
    • 0.9.0
    • None
    • Spark Integration

    Description

      This is the main task for supporting spark sql for hudi, including the DDL、DML and Hoodie CLI command.

      Attachments

        Issue Links

          1.
          Basic implementation Of Spark Sql Support Sub-task Resolved pengzhiwei
          2.
          Support SQL with spark3 Sub-task Closed tao meng
          3.
          Support AlterCommand For Hoodie Sub-task Resolved pengzhiwei
          4.
          Support Clustering and Metatable for SQL performance Sub-task Open Unassigned
          5.
          Support Cluster By In Create Table Sub-task Open Unassigned
          6.
          Support Create Index In Create Table Sub-task Open Unassigned
          7.
          Support Hoodie CLI Command In Spark SQL Sub-task Open Yann Byron
          8.
          Add Commit Time Prefix To The UuidKeyGenerator For Insert Only Sub-task Open pengzhiwei
          9.
          [SQL] Spark Sql Support For The Exists Hoodie Table Sub-task Resolved pengzhiwei
          10.
          Support Spark 3.1(Duplicated) Sub-task Closed Unassigned
          11.
          Upgrading Spark3 To 3.1 Sub-task Resolved Yann Byron
          12.
          Support Truncate Table For Hoodie Sub-task Resolved pengzhiwei
          13.
          MergeInto Support Partial Update For COW Sub-task Resolved pengzhiwei
          14.
          Support Delete/Update Non-Pk Table Sub-task Open Yann Byron
          15.
          Performance testing/certification of key SQL DMLs Sub-task Closed Raymond Xu
          16.
          Enable Hive Sync When Spark Enable Hive Meta For Spark Sql Sub-task Resolved pengzhiwei
          17.
          [SQL] Add Doc For Spark Sql Integrates With Hudi Sub-task Resolved pengzhiwei
          18.
          Support Alter table drop column Sub-task Open Yann Byron
          19.
          Support Compaction Command For Spark Sql Sub-task Resolved pengzhiwei
          20.
          [SQL] Support Bulk Insert For Spark Sql Sub-task Resolved pengzhiwei
          21.
          Missing PrimaryKey In Hoodie Properties For CTAS Table Sub-task Resolved pengzhiwei
          22.
          [SQL] Functionality testing with Spark 2 Sub-task Closed Sagar Sumit
          23.
          [SQL] Test catalog integration Sub-task Closed Sagar Sumit
          24.
          [SQL] MERGE INTO fails with table having nested struct Sub-task Resolved pengzhiwei
          25.
          [SQL] Hive sync is not working Sub-task Resolved pengzhiwei
          26.
          MERGE INTO works only ON primary key Sub-task Closed Yann Byron
          27.
          [SQL] Changing index type fails Sub-task Resolved sivabalan narayanan
          28.
          [SQL] Bulk insert support for tables w/ primary key Sub-task Resolved pengzhiwei
          29.
          [SQL] Fix Exception Cause By Table Name Case Sensitivity For Append Mode Write Sub-task Resolved pengzhiwei
          30.
          Upgrade hoodie table to 0.9.0 Sub-task Resolved sivabalan narayanan
          31.
          Insert for an already existing record throws DuplicateKeyException with primary keyed spark sql table Sub-task Resolved pengzhiwei
          32.
          Support Clustering Command For Spark Sql Sub-task Open Unassigned
          33.
          Support delete partitions via alter table Sub-task Reopened Unassigned
          34.
          MERGE INTO doesn't work for tables created using CTAS Sub-task Resolved pengzhiwei
          35.
          [SQL]Support referencing subquery with column aliases by table alias in merge into Sub-task Closed 董可伦
          36.
          Fix the exception for mergeInto when the primaryKey and preCombineField of source table and target table differ in case only Sub-task Resolved 董可伦
          37.
          Introduce config to allow users to control case-sensitivity in column projections #431 Sub-task Open Unassigned
          38.
          Support Multipath query for HoodieFileIndex Sub-task Open pengzhiwei
          39.
          Support drop partitions SQL Sub-task Resolved Yann Byron
          40.
          Support show partitions SQL Sub-task Resolved Yann Byron
          41.
          Delete data is not working with 0.9.0 and pySpark Sub-task Open Unassigned
          42.
          use commit_time in the WHERE STATEMENT to optimize the incremental query Sub-task Open David_Liang
          43.
          Create Table If Not Exists Failed After Alter Table Sub-task Resolved pengzhiwei
          44.
          Support point lookup queries leveraging the bloom filter indexing Sub-task Open Unassigned
          45.
          Fix Spark version info for hudi table CTAS from another hudi table Sub-task Open Unassigned
          46.
          `create table if not exists` should print message instead of throwing error Sub-task Open Unassigned

          Activity

            People

              biyan900116@gmail.com Yann Byron
              pzw2018 pengzhiwei
              Raymond Xu, Shaofeng Li
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: