Uploaded image for project: 'CarbonData'
  1. CarbonData
  2. CARBONDATA-619

Compaction API for Spark 2.1 : Issue in compaction type

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0-incubating
    • Fix Version/s: 1.0.0-incubating
    • Component/s: data-query
    • Labels:
      None
    • Environment:
      Spark 2.1

      Description

      Compaction API for Spark 2.1 : Compaction is successful on the segment ids irrespective of the compaction types which are( Minor and Major) i.e The compaction was successful for type other than Minor/Major

      I created a table with name carbon_table and made 5 loads into the table with the threshold of (4,3)

      cc.sql("create table carbon_table(id int,name string) stored by 'carbondata'")

      cc.sql("LOAD DATA LOCAL INPATH 'hdfs://hadoop-master:54311/data/employee.csv'
      INTO TABLE carbon_table options('FILEHEADER'='id,name')")

      For instance
      Compaction.compaction(spark,"default","carbon_table","xyz")
      here
      default is the database name
      carbon_table is the table name
      xyz is the compaction type

      I invoked the above method to compact the segments and it was executed successfully and compacted the segment ids. On show segment It produced the result with success for compaction . I have attached the screen shot for the same.

        Attachments

        1. employee.csv
          0.0 kB
          DEEPAK MEHRA
        2. compaction_result.png
          54 kB
          DEEPAK MEHRA

          Issue Links

            Activity

              People

              • Assignee:
                ravikiran23 ravikiran
                Reporter:
                deepakmehra DEEPAK MEHRA
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m