Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-440

Provide Update/Delete functionality support in CarbonData

    XMLWordPrintableJSON

    Details

      Description

      Currently, CarbonData does not support modification of existing rows in the table. This is a major limitation for many real world desirable use cases in data warehousing, such as slow changing dimension tables, data correction of fact tables or data cleanup, etc. Many users want to be able to update and delete rows from the CarbonData table.

      Following are some high level design goals to support this functionality,
      1. Provide a standard SQL interface to perform Update and Delete operations.
      2. Perform Update and Delete operations on CarbonData table without having to rewrite the entire CarbonData block (file) by making use of differential files (a.k.a delta files).
      3. After Update or Delete operation, CarbonData readers should skip deleted records and read updated records seamlessly without having to modify user applications.
      4. Perform compaction of Update and Delete delta files to maintain adequate query performance

        Attachments

        There are no Sub-Tasks for this issue.

          Activity

            People

            • Assignee:
              aniketadnaik Aniket Adnaik
              Reporter:
              aniketadnaik Aniket Adnaik
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 13h
                13h