Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7299

saving Oracle-source DataFrame to Hive changes scale

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.1
    • 1.4.0
    • SQL
    • None

    Description

      When I load data from Oracle, save it to a table, then select from it, the scale is changed.

      For example, I have a column defined as NUMBER(12, 2). I insert 19999 into the column. When I write that to a table and select from it, the result is 199.99.

      Some databases (e.g. H2) will return this as 19999.00, but Oracle returns it as 19999. I believe that when the file is written out to parquet, the scale information is taken from the schema, not the value. In an Oracle (at least) JDBC DataFrame, the scale may be different from row to row.

      Attachments

        Activity

          People

            viirya L. C. Hsieh
            kgeis Ken Geis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: