Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8532

In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.4.0
    • 1.4.1, 1.5.0
    • SQL
    • None

    Description

      Although users can use df.write.mode("overwrite") to specify the mode, when save/saveAsTable/json/parquet/jdbc is called, this mode will be overridden. For example, the implementation of json method is

      def json(self, path, mode="error"):
        self._jwrite.mode(mode).json(path)
      

      If users only call json("path"), the mode will be "error" instead of the mode specified in the mode method.

      Attachments

        Activity

          People

            yhuai Yin Huai
            yhuai Yin Huai
            Davies Liu Davies Liu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: