Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16469

Parquet timestamp table property is not always taken into account

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The parquet timestamp timezone property is currently copied over into the JobConf in the FetchOperator, but this may be too late for some execution paths.
      We should:
      1 - copy the property over earlier
      2 - set the default value on the JobConf if no property is set, and fail in the ParquetRecordReader if the property is missing from the JobConf

      We should add extra validations for the cases when:

      • the property was not set by accident on the JobConf (unexpected execution path)
      • an incorrect/invalid timezone id is being set on the table

        Attachments

        1. HIVE-16469.05.patch
          71 kB
          Barna Zsombor Klara
        2. HIVE-16469.04.patch
          71 kB
          Barna Zsombor Klara
        3. HIVE-16469.03.patch
          68 kB
          Barna Zsombor Klara
        4. HIVE-16469.02.patch
          57 kB
          Barna Zsombor Klara
        5. HIVE-16469.01.patch
          53 kB
          Barna Zsombor Klara

          Issue Links

            Activity

              People

              • Assignee:
                zsombor.klara Barna Zsombor Klara
                Reporter:
                zsombor.klara Barna Zsombor Klara
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: