XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • Tests
    • None

    Description

      After analyzing the latest test results https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1200/testReport/org.apache.hadoop.hive.cli/
      there are many Minimr tests that are slow individually. There are also some tests that does not have to run in minimr, some tests are testing hdfs related features that can be migrated to MiniLlap (some are already in MiniSpark). Since we have removed hadoop20 now some MiniMr tests can be moved to TestCliDriver. We should keep the absolute minimum number of tests in minimr and move other slow test to minillap (spark already has most of these tests).

      These are the individual test runtimes for minimr tests

      QFiles	TestMinimrCliDriver elapsed time (Build #1055)
      infer_bucket_sort_reducers_power_two.q	356.211
      bucket5.q	257.589
      infer_bucket_sort_bucketed_table.q	251.557
      bucketizedhiveinputformat.q	249.755
      infer_bucket_sort_map_operators.q	249.484
      index_bitmap_auto.q	212.9
      infer_bucket_sort_dyn_part.q	204.401
      skewjoin_onesideskew.q	198.428
      index_bitmap3.q	194.221
      truncate_column_buckets.q	193.966
      auto_sortmerge_join_16.q	171.094
      schemeAuthority.q	170.839
      leftsemijoin_mr.q	150.493
      join_acid_non_acid.q	143.232
      empty_dir_in_table.q	136.463
      bucketmapjoin6.q	129.786
      reduce_deduplicate.q	116.455
      exchgpartition2lel.q	115.678
      bucket_many.q	104.316
      infer_bucket_sort_merge.q	100.976
      quotedid_smb.q	95.801
      external_table_with_space_in_location_path.q	80.265
      schemeAuthority2.q	72.64
      root_dir_external_table.q	72.171
      parallel_orderby.q	71.662
      disable_merge_for_bucketing.q	69.25
      bucket_num_reducers.q	67.618
      remote_script.q	67.029
      groupby2.q	66.72
      bucket4.q	66.02
      udf_using.q	64.985
      file_with_header_footer.q	42.986
      infer_bucket_sort_num_buckets.q	38.795
      scriptfile1.q	35.317
      bucket6.q	34.963
      non_native_window_udf.q	34.129
      join1.q	33.831
      bucketmapjoin7.q	33.71
      bucket_num_reducers2.q	30.645
      list_bucket_dml_10.q	25.751
      insert_dir_distcp.q	25.723
      uber_reduce.q	16.481
      load_hdfs_file_with_space_in_the_name.q	3.899
      table_nonprintable.q	2.097
      load_fs2.q	1.947
      input16_cc.q	0.916
      temp_table_external.q	0.751
      import_exported_table.q	0.508
      scriptfile1_win.q	0.119
      

      Attachments

        1. HIVE-14767.1.patch
          183 kB
          Prasanth Jayachandran
        2. HIVE-14767.2.patch
          200 kB
          Prasanth Jayachandran
        3. HIVE-14767.3.patch
          183 kB
          Prasanth Jayachandran
        4. HIVE-14767.4.patch
          171 kB
          Prasanth Jayachandran

        Activity

          People

            prasanth_j Prasanth Jayachandran
            prasanth_j Prasanth Jayachandran
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: