[CARBONDATA-2082] Timeseries pre-aggregate table should support the blank space

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.0
• Component/s: core, spark-integration
• External issue URL: https://issues.apache.org/jira/browse/CARBONDATA-1516
• Assignee: Bo Xu
• Reporter: Bo Xu
• Created: 26/Jan/18 03:28
• Updated: 02/Feb/18 14:38
• Resolved: 02/Feb/18 13:10

Description

timeseries pre-aggregate table should support the blank space

1.scenario 1

   test("test timeseries create table 35: support event_time and granularity key with space") {
	    sql("DROP DATAMAP IF EXISTS agg1_month ON TABLE maintable")
	    sql(
	      s"""CREATE DATAMAP agg1_month ON TABLE mainTable
	         |USING '$timeSeries'
	         |DMPROPERTIES (
	         |   'event_time '=' dataTime',
	         |   'MONTH_GRANULARITY '='1')
	         |AS SELECT dataTime, SUM(age) FROM mainTable
	         |GROUP BY dataTime
	        """.stripMargin)
	    checkExistence(sql("SHOW TABLES"), true, "maintable_agg1_month")
	  }

problem: NPE

	java.lang.NullPointerException was thrown.
	java.lang.NullPointerException
		at org.apache.spark.sql.execution.command.timeseries.TimeSeriesUtil$.validateTimeSeriesEventTime(TimeSeriesUtil.scala:50)
		at org.apache.spark.sql.execution.command.preaaggregate.CreatePreAggregateTableCommand.processMetadata(CreatePreAggregateTableCommand.scala:104)
		at org.apache.spark.sql.execution.command.datamap.CarbonCreateDataMapCommand.processMetadata(CarbonCreateDataMapCommand.scala:75)
		at org.apache.spark.sql.execution.command.AtomicRunnableCommand.run(package.scala:84)
		at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)

2.scenario 2

		  test("test timeseries create table 36: support event_time and granularity key with space") {
		    sql("DROP DATAMAP IF EXISTS agg1_month ON TABLE maintable")
		    sql(
		      s"""CREATE DATAMAP agg1_month ON TABLE mainTable
		         |USING '$timeSeries'
		         |DMPROPERTIES (
		         |   'event_time '='dataTime',
		         |   'MONTH_GRANULARITY '=' 1')
		         |AS SELECT dataTime, SUM(age) FROM mainTable
		         |GROUP BY dataTime
		        """.stripMargin)
		    checkExistence(sql("SHOW TABLES"), true, "maintable_agg1_month")
		  }
	

problem:

	Granularity only support 1
	org.apache.carbondata.spark.exception.MalformedDataMapCommandException: Granularity only support 1
		at org.apache.spark.sql.execution.command.timeseries.TimeSeriesUtil$.getTimeSeriesGranularityDetails(TimeSeriesUtil.scala:118)
		at org.apache.spark.sql.execution.command.datamap.CarbonCreateDataMapCommand.processMetadata(CarbonCreateDataMapCommand.scala:58)
		at org.apache.spark.sql.execution.command.AtomicRunnableCommand.run(package.scala:84)
		at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
		at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
		at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
		at org.apache.spark.sql.Dataset.<init>(Dataset.scala:183)
		at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:68)
		at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:632)

Issue Links

• links to: GitHub Pull Request #1868
• links to: GitHub Pull Request #1902