[HBASE-22355] address Spark deprecations in hbase-connectors - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Minor
Resolution: Unresolved
Affects Version/s: 1.0.0
Fix Version/s: None
Component/s: hbase-connectors
Labels:
None

Description

back on the the thread in https://www.mail-archive.com/dev@hbase.apache.org/msg54431.html address the recommendations and set minimum Spark version for connectors, specify explicit version in Readme.

[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableCatalog.scala:289: warning: @deprecated now takes two arguments; see the scaladoc.
[WARNING] @deprecated("Please use new json format to define HBaseCatalog")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:168: warning: class HTableDescriptor in package hbase is deprecated: see corresponding Javadoc for more information.
[WARNING] val tableDesc = new HTableDescriptor(tName)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:170: warning: constructor HColumnDescriptor in class HColumnDescriptor is deprecated: see corresponding Javadoc for more information.
[WARNING] val cf = new HColumnDescriptor(x.getBytes())
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:170: warning: class HColumnDescriptor in package hbase is deprecated: see corresponding Javadoc for more information.
[WARNING] val cf = new HColumnDescriptor(x.getBytes())
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableCatalog.scala:243: warning: method convert in object HBaseTableCatalog is deprecated: Please use new json format to define HBaseCatalog
[WARNING] val parameters = convert(params)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:162: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] case (Some(Bound(a, b)), Some(Bound(c, d))) => new Scan(a, c)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:163: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] case (None, Some(Bound(c, d))) => new Scan(Array[Byte](), c)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:164: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] case (Some(Bound(a, b)), None) => new Scan(a)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:246: warning: method setTimeStamp in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] case (q: Scan, Some(ts), None, None) => q.setTimeStamp(ts)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:247: warning: method setTimeStamp in class Get is deprecated: see corresponding Javadoc for more information.
[WARNING] case (q: Get, Some(ts), None, None) => q.setTimeStamp(ts)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:260: warning: method setMaxVersions in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] case q: Scan => q.setMaxVersions(relation.maxVersions.get)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:261: warning: method setMaxVersions in class Get is deprecated: see corresponding Javadoc for more information.
[WARNING] case q: Get => q.setMaxVersions(relation.maxVersions.get)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/AvroSource.scala:105: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
[WARNING] val sqlContext = new SQLContext(sc)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/AvroSource.scala:129: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("ExampleAvrotable")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala:86: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
[WARNING] val sqlContext = new SQLContext(sc)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/HBaseSource.scala:76: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
[WARNING] val sqlContext = new SQLContext(sc)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/HBaseSource.scala:105: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("table1")
[WARNING] ^
[WARNING] 17 warnings found
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:209: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
[WARNING] sqlContext = new SQLContext(sc)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:211: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:214: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseTable1")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:227: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:230: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseTable2")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:591: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:594: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseNonExistingTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:616: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:619: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseFactColumnTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:642: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:645: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseIntTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:673: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:676: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseIntWrongTypeTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:705: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:708: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseIntWrongTypeTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:723: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] val df = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:729: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] val df1 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:736: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] val df2 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:743: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] val df3 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:762: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:765: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseZTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:793: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
[WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:797: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("hbaseNoPushDownTmp")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:848: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("table0")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:995: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
[WARNING] df.registerTempTable("avrotable")
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseCatalogSuite.scala:87: warning: method convert in object HBaseTableCatalog is deprecated: Please use new json format to define HBaseCatalog
[WARNING] val map = HBaseTableCatalog.convert(m)
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala:337: warning: method setStartRow in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] scan.setStartRow(Bytes.toBytes("scan2"))
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala:338: warning: method setStopRow in class Scan is deprecated: see corresponding Javadoc for more information.
[WARNING] scan.setStopRow(Bytes.toBytes("scan4_"))
[WARNING] ^
[WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/PartitionFilterSuite.scala:77: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
[WARNING] sqlContext = new SQLContext(sc)
[WARNING] ^
[WARNING] 29 warnings found

Attachments

Activity

People

Assignee:: Artem Ervits

Reporter:: Artem Ervits

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 02/May/19 19:26

Updated:: 02/May/19 19:36