Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9134 Uber JIRA to track HOS performance work
  3. HIVE-9153

Perf enhancement on CombineHiveInputFormat and HiveInputFormat

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0
    • Spark
    • None

    Description

      The default InputFormat is CombineHiveInputFormat and thus HOS uses this. However, Tez uses HiveInputFormat. Since tasks are relatively cheap in Spark, it might make sense for us to use HiveInputFormat as well. We should evaluate this on a query which has many input splits such as select count(*) from store_sales where something is not null.

      Attachments

        1. screenshot.PNG
          104 kB
          Rui Li
        2. HIVE-9153.3.patch
          4 kB
          Rui Li
        3. HIVE-9153.2.patch
          6 kB
          Rui Li
        4. HIVE-9153.1-spark.patch
          6 kB
          Rui Li
        5. HIVE-9153.1-spark.patch
          6 kB
          Brock Noland

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-4921 TaskSetManager mistakenly returns PROCESS_LOCAL for NO_PREF tasks

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              lirui Rui Li
              brocknoland Brock Noland
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: