Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24768 Have a built-in AVRO data source implementation
  3. SPARK-24805

Don't ignore files without .avro extension by default

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.1
    • 2.4.0
    • SQL
    • None

    Description

      Currently to read files without .avro extension, users have to set the flag avro.mapred.ignore.inputs.without.extension to false (by default it is true). The ticket aims to change the default value to false. The reasons to do that are:

      • Other systems can create avro files without extensions. When users try to read such files, they get just partitial results silently. The behaviour may confuse users.
      • Current behavior is different behavior from another supported datasource CSV and JSON.

      Attachments

        Activity

          People

            maxgekk Max Gekk
            maxgekk Max Gekk
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: