Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22355

address Spark deprecations in hbase-connectors

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: hbase-connectors
    • Labels:
      None

      Description

      back on the the thread in https://www.mail-archive.com/dev@hbase.apache.org/msg54431.html address the recommendations and set minimum Spark version for connectors, specify explicit version in Readme.

      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableCatalog.scala:289: warning: @deprecated now takes two arguments; see the scaladoc.
      [WARNING] @deprecated("Please use new json format to define HBaseCatalog")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:168: warning: class HTableDescriptor in package hbase is deprecated: see corresponding Javadoc for more information.
      [WARNING] val tableDesc = new HTableDescriptor(tName)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:170: warning: constructor HColumnDescriptor in class HColumnDescriptor is deprecated: see corresponding Javadoc for more information.
      [WARNING] val cf = new HColumnDescriptor(x.getBytes())
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/DefaultSource.scala:170: warning: class HColumnDescriptor in package hbase is deprecated: see corresponding Javadoc for more information.
      [WARNING] val cf = new HColumnDescriptor(x.getBytes())
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableCatalog.scala:243: warning: method convert in object HBaseTableCatalog is deprecated: Please use new json format to define HBaseCatalog
      [WARNING] val parameters = convert(params)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:162: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] case (Some(Bound(a, b)), Some(Bound(c, d))) => new Scan(a, c)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:163: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] case (None, Some(Bound(c, d))) => new Scan(Array[Byte](), c)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:164: warning: constructor Scan in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] case (Some(Bound(a, b)), None) => new Scan(a)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:246: warning: method setTimeStamp in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] case (q: Scan, Some(ts), None, None) => q.setTimeStamp(ts)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:247: warning: method setTimeStamp in class Get is deprecated: see corresponding Javadoc for more information.
      [WARNING] case (q: Get, Some(ts), None, None) => q.setTimeStamp(ts)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:260: warning: method setMaxVersions in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] case q: Scan => q.setMaxVersions(relation.maxVersions.get)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/datasources/HBaseTableScanRDD.scala:261: warning: method setMaxVersions in class Get is deprecated: see corresponding Javadoc for more information.
      [WARNING] case q: Get => q.setMaxVersions(relation.maxVersions.get)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/AvroSource.scala:105: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
      [WARNING] val sqlContext = new SQLContext(sc)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/AvroSource.scala:129: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("ExampleAvrotable")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/DataType.scala:86: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
      [WARNING] val sqlContext = new SQLContext(sc)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/HBaseSource.scala:76: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
      [WARNING] val sqlContext = new SQLContext(sc)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/main/scala/org/apache/hadoop/hbase/spark/example/datasources/HBaseSource.scala:105: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("table1")
      [WARNING] ^
      [WARNING] 17 warnings found
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:209: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
      [WARNING] sqlContext = new SQLContext(sc)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:211: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:214: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseTable1")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:227: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:230: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseTable2")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:591: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:594: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseNonExistingTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:616: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:619: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseFactColumnTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:642: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:645: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseIntTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:673: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:676: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseIntWrongTypeTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:705: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:708: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseIntWrongTypeTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:723: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] val df = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:729: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] val df1 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:736: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] val df2 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:743: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] val df3 = sqlContext.load("org.apache.hadoop.hbase.spark.HBaseTestSource",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:762: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:765: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseZTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:793: warning: method load in class SQLContext is deprecated: Use read.format(source).options(options).load() instead.
      [WARNING] df = sqlContext.load("org.apache.hadoop.hbase.spark",
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:797: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("hbaseNoPushDownTmp")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:848: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("table0")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/DefaultSourceSuite.scala:995: warning: method registerTempTable in class Dataset is deprecated: Use createOrReplaceTempView(viewName) instead.
      [WARNING] df.registerTempTable("avrotable")
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseCatalogSuite.scala:87: warning: method convert in object HBaseTableCatalog is deprecated: Please use new json format to define HBaseCatalog
      [WARNING] val map = HBaseTableCatalog.convert(m)
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala:337: warning: method setStartRow in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] scan.setStartRow(Bytes.toBytes("scan2"))
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/HBaseContextSuite.scala:338: warning: method setStopRow in class Scan is deprecated: see corresponding Javadoc for more information.
      [WARNING] scan.setStopRow(Bytes.toBytes("scan4_"))
      [WARNING] ^
      [WARNING] /Users/aervits/NetBeansProjects/hbase-connectors/spark/hbase-spark/src/test/scala/org/apache/hadoop/hbase/spark/PartitionFilterSuite.scala:77: warning: constructor SQLContext in class SQLContext is deprecated: Use SparkSession.builder instead
      [WARNING] sqlContext = new SQLContext(sc)
      [WARNING] ^
      [WARNING] 29 warnings found
      

        Attachments

          Activity

            People

            • Assignee:
              dbist13 Artem Ervits
              Reporter:
              dbist13 Artem Ervits
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: