[CARBONDATA-619] Compaction API for Spark 2.1 : Issue in compaction type

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0-incubating
• Fix Version/s: 1.0.0-incubating
• Component/s: data-query
• Environment: Spark 2.1
• Assignee: ravikiran
• Reporter: DEEPAK MEHRA
• Created: 10/Jan/17 12:59
• Updated: 17/Jan/17 20:08
• Resolved: 17/Jan/17 20:08

Description

Compaction API for Spark 2.1 : Compaction is successful on the segment ids irrespective of the compaction types which are( Minor and Major) i.e The compaction was successful for type other than Minor/Major

I created a table with name carbon_table and made 5 loads into the table with the threshold of (4,3)

cc.sql("create table carbon_table(id int,name string) stored by 'carbondata'")

cc.sql("LOAD DATA LOCAL INPATH 'hdfs://hadoop-master:54311/data/employee.csv'
INTO TABLE carbon_table options('FILEHEADER'='id,name')")

For instance
Compaction.compaction(spark,"default","carbon_table","xyz")
here
default is the database name
carbon_table is the table name
xyz is the compaction type

I invoked the above method to compact the segments and it was executed successfully and compacted the segment ids. On show segment It produced the result with success for compaction . I have attached the screen shot for the same.

Issue Links

• links to: GitHub Pull Request #543