Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
-
spark 2.1, hive 1.2.1
Description
Selecting columns non sequentially(out of the order) fails to render output
Steps to reproduce:
1) In Spark Shell :
a) Create Table -
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.CarbonSession._
val carbon = SparkSession.builder().config(sc.getConf).getOrCreateCarbonSession("hdfs://localhost:54310/opt/data")
scala> carbon.sql(" create table abc(id int, name string) stored by 'carbondata' ").show
b) Load Data -
scala> carbon.sql(""" load data inpath 'hdfs://localhost:54310/Files/abc.csv' into table abc """ ).show
2) In Hive :
a) Add Jars -
add jar /home/neha/incubator-carbondata/assembly/target/scala-2.11/carbondata_2.11-1.1.0-incubating-SNAPSHOT-shade-hadoop2.7.2.jar;
add jar /opt/spark-2.1.0-bin-hadoop2.7/jars/spark-catalyst_2.11-2.1.0.jar;
add jar /home/neha/incubator-carbondata/integration/hive/carbondata-hive-1.1.0-incubating-SNAPSHOT.jar;
b) Create Table -
create table abc(id int,name string);
c) Alter location -
hive> alter table abc set LOCATION 'hdfs://localhost:54310/opt/data/default/abc' ;
d) Set Properties -
set hive.mapred.supports.subdirectories=true;
set mapreduce.input.fileinputformat.input.dir.recursive=true;
d) Alter FileFormat -
alter table abc set FILEFORMAT
INPUTFORMAT "org.apache.carbondata.hive.MapredCarbonInputFormat"
OUTPUTFORMAT "org.apache.carbondata.hive.MapredCarbonOutputFormat"
SERDE "org.apache.carbondata.hive.CarbonHiveSerDe";
e) Queries -
hive> select id from abc; //Works Fine(Column in order)
hive> select name from abc; //Doesn't Work(Column out of order)
hive> select id,name from abc; //Works Fine(Columns in order)
hive> select name,id from abc; //Doesn't Work(Columns out of order)
Expected output : Query - hive> select name,id from abc;
display data of the columns specified.
Actual output : Query - hive> select name,id from abc;
OK
Failed with exception java.io.IOException:java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Long
Time taken: 0.079 seconds
