Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8584

Setting hive.exec.orc.default.compress to ZLIB will lead to orc file size delta byte(s) shorter on Windows than Linux

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Invalid
    • 0.14.0
    • None
    • None
    • None
    • Windows

    Description

      repo steps:
      1. run query orc_analyze.q
      2. hive --orcfiledump <target_orc_file_generated>

      run 1 and 2 on PST timezone on Linux, and one more time on other timezone e.g. CST on Windows.
      Compare two target orc file dumping. Windows orc file is 1 byte shorter than Linux one.

      That's the case even if running 1 and 2 on Windows for different timezones, however, no problem on Linux.

      The issue only exists by using ZLIB mode, eventually OS native compression lib is used.

      Attachments

        1. HIVE-8584.1.patch
          1 kB
          Xiaobing Zhou
        2. orc_analyze.q
          0.9 kB
          Xiaobing Zhou
        3. orc-win-none-1.dump
          3 kB
          Xiaobing Zhou
        4. orc-win-none-2.dump
          3 kB
          Xiaobing Zhou
        5. orc-win-snappy-1.dump
          3 kB
          Xiaobing Zhou
        6. orc-win-snappy-2.dump
          3 kB
          Xiaobing Zhou
        7. orc-win-zlib-1.dump
          3 kB
          Xiaobing Zhou
        8. orc-win-zlib-2.dump
          3 kB
          Xiaobing Zhou

        Activity

          People

            xiaobingo Xiaobing Zhou
            xiaobingo Xiaobing Zhou
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: