[SPARK-30490] Eliminate warnings in Avro datasource

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Max Gekk
• Created: 11/Jan/20 11:54
• Updated: 15/Jan/20 12:15
• Resolved: 15/Jan/20 12:13

Description

Here is the list of deprecation warning in Avro:

/Users/maxim/proj/eliminate-warning/external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala
    Warning:Warning:line (98)value ignoreExtension in class AvroOptions is deprecated (since 3.0): Use the general data source option pathGlobFilter for filtering file names
      if (parsedOptions.ignoreExtension || file.filePath.endsWith(".avro")) {
/Users/maxim/proj/eliminate-warning/external/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala
    Warning:Warning:line (55)value ignoreExtension in class AvroOptions is deprecated (since 3.0): Use the general data source option pathGlobFilter for filtering file names
        inferAvroSchemaFromFiles(files, conf, parsedOptions.ignoreExtension,
/Users/maxim/proj/eliminate-warning/external/avro/src/main/scala/org/apache/spark/sql/v2/avro/AvroPartitionReaderFactory.scala
    Warning:Warning:line (64)value ignoreExtension in class AvroOptions is deprecated (since 3.0): Use the general data source option pathGlobFilter for filtering file names
    if (parsedOptions.ignoreExtension || partitionedFile.filePath.endsWith(".avro")) {

 

Issue Links

• is related to: SPARK-30171 Eliminate warnings: part 2
• relates to: SPARK-30505 Deprecate Avro option `ignoreExtension` in a doc
• links to: GitHub Pull Request #27174