Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2125

Improve perf when reading timestamps from parquet files written by hive

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Duplicate
    • Impala 2.2
    • None
    • Backend
    • None

    Description

      This is for tracking purposes. The improvement is already committed – 29de99c9d25c49b73488d2f75bc3644ae9ff9325.

      When using the flag -convert_legacy_hive_parquet_utc_timestamps=true, depending on the query the runtime may be 10x longer (possibly more). The commit above inlines some function calls which improves the 10x case to 5x.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-3316 convert_legacy_hive_parquet_utc_timestamps=true makes reading parquet tables 30x slower

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              caseyc casey
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: