[CARBONDATA-2001] Unable to save a dataframe result as carbondata streaming table

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.0
• Component/s: spark-integration
• Environment: spark-2.1
• Assignee: anubhav tarar
• Reporter: anubhav tarar
• Created: 08/Jan/18 06:54
• Updated: 19/Jan/18 13:44
• Resolved: 19/Jan/18 13:44

Description

1.create carbonsession

import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.CarbonSession._
val carbon = SparkSession.builder().config(sc.getConf) .getOrCreateCarbonSession("hdfs://localhost:54311/newCarbonStore","/tmp"

2.create a dataframe with carbonsession
import carbon.sqlContext.implicits._
carbon.sql("drop table if exists streamingtable");
val df =carbon.sparkContext.parallelize(1 to 5).toDF("colId")

3.register dataframe as carbon streaming table

df.write.format("carbondata").option("tableName","streamingTable").option("streaming","true").mode(SaveMode.Overwrite).save

4,desc formatted the table

carbon.sql("describe formatted streamingTable").show(100)
--------------------------------------------------------

col_name

data_type

comment

--------------------------------------------------------

colid ...	int ...	MEASURE,null ...
...	...	...
##Detailed Table ...	...	...
Database Name ...	default ...	...
Table Name ...	streamingtable ...	...
CARBON Store Path...	hdfs://localhost:...	...
Comment ...	...	...
Table Block Size ...	1024 MB ...	...
Table Data Size ...	316 ...	...
Table Index Size ...	283 ...	...
Last Update Time ...	1515393447642 ...	...
SORT_SCOPE ...	LOCAL_SORT ...	LOCAL_SORT ...
Streaming ...	false ...	...
SORT_SCOPE ...	LOCAL_SORT ...	LOCAL_SORT ...
...	...	...
##Detailed Column...	...	...
ADAPTIVE ...	...	...
SORT_COLUMNS ...	...	...

--------------------------------------------------------

here property streaming is false it should be true

Issue Links

• links to: GitHub Pull Request #1774