Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22942

Replace PTest with an alternative

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: None

      Description

      I never opened a jira about this...but it might actually help collect ideas and actually start going somewhere sooner than later

      Right now we maintain the ptest2 project inside Hive to be able to run Hive tests in a distributed fashion...the backstab of this solution is that we are putting much effort into maintaining a distributed test execution framework...

      I think it would be better if we could find an off the shelf solution for the task and migrate to that instead of putting more efforts into the ptest framework

      some info/etc about how it compares to existing one:
      https://docs.google.com/document/d/1dhL5B-eBvYNKEsNV3kE6RrkV5w-LtDgw5CtHV5pdoX4/edit#heading=h.e51vlxui3e6n

        Attachments

        1. HIVE-22942.01.patch
          25 kB
          Zoltan Haindrich

          Issue Links

          1.
          CliDriver leaves the session id in the threadname on failure Sub-task Resolved Zoltan Haindrich  
          2.
          UDAFExampleMaxMinNUtil.sortedMerge must be able to handle all inputs Sub-task Resolved Zoltan Haindrich  
          3.
          PostExecOrcFileDump listing order may depend on the underlying filesystem Sub-task Resolved Zoltan Haindrich  
          4.
          Reduce MiniDruidCluster memory requeirements Sub-task Resolved Zoltan Haindrich  
          5.
          Increase timeout for some tez tests Sub-task Resolved Zoltan Haindrich  
          6.
          avro-mapred should not pull in org.mortbay.jetty Sub-task Resolved Zoltan Haindrich  
          7.
          Prevent infinite loop in TestJdbcWithMiniLlapArrow Sub-task Resolved Zoltan Haindrich  
          8.
          Scheduled query related qtests may not finish before it's expected Sub-task Resolved Zoltan Haindrich  
          9.
          Provide a way to have only a selection of datasets loaded Sub-task Resolved Zoltan Haindrich  
          10.
          Change spark related tests to be optional Sub-task Resolved Zoltan Haindrich  
          11.
          Update some tests to better tolerate a more busy environment Sub-task Resolved Zoltan Haindrich  
          12.
          Tolerate in tests that metastore is not removing rows from the skewed_string_list_values table Sub-task Resolved Zoltan Haindrich  
          13.
          Add qsplits profile Sub-task Resolved Zoltan Haindrich

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          14.
          schq_ingest may run twice during a test execution Sub-task Resolved Zoltan Haindrich  
          15.
          QueryDisplay must be threadsafe Sub-task Resolved Zoltan Haindrich  
          16.
          Add qoption to disable qtests Sub-task Resolved Zoltan Haindrich

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          17.
          Many fixes and improvements to stabilize tests Sub-task Resolved Zoltan Haindrich

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          18.
          Upgrade to Kudu 1.12 Sub-task Resolved Zoltan Haindrich  
          19.
          Running masking_mv.q breaks strict_pruning.q Sub-task Patch Available Zoltan Haindrich  
          20.
          stats_list_bucketing breaks sysdb output Sub-task Open Zoltan Haindrich  
          21.
          Use junit5 to execute tests Sub-task Patch Available Zoltan Haindrich

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10m
          22.
          Provide a way to include/exclude a set of tests Sub-task Open Zoltan Haindrich  
          23.
          Rows are not removed from the skewed_string_list_values table Sub-task Open Unassigned  
          24.
          TestJdbcGenericUDTFGetSplits2#testGenericUDTFOrderBySplitCount1 is flaky Sub-task Open Unassigned  
          25.
          TestScheduledReplicationScenarios#testAcidTablesReplLoadBootstrapIncr is unstable Sub-task Open Unassigned  
          26.
          TestBeeLineWithArgs#testRowsAffected is unstable Sub-task Open Unassigned  
          27.
          TestSessionManagerMetrics#testAbandonedSessionMetrics is unstable Sub-task Open Unassigned  
          28.
          TransactionalKafkaWriterTest#writerFencedOut is unstable Sub-task Open Unassigned  
          29.
          TestDbNotificationListener#sqlInsertPartition is unstable Sub-task Open Unassigned  
          30.
          strict_pruning_2 is unstable Sub-task Open Unassigned  
          31.
          TestMiniDruidCliDriver#testCliDriver[druid_materialized_view_rewrite_ssb] is flaky Sub-task Open Unassigned  
          32.
          TestTriggersTezSessionPoolManager sometimes exits the JVM Sub-task Open Unassigned  
          33.
          TestNewGetSplitsFormat is unstable Sub-task Open Unassigned  
          34.
          TestAcidTxnCleanerService is unstable Sub-task Patch Available Peter Varga  
          35.
          TestJdbcWithServiceDiscovery is unstable Sub-task Open Unassigned  
          36.
          TestActivePassiveHA is unstable Sub-task Open Unassigned  
          37.
          TestJdbcWithMiniLlapVectorArrow is unstable Sub-task Open Unassigned  
          38.
          TestMiniLlapCliDriver.testCliDriver[merge_test_dummy_operator] is unstable Sub-task Open Unassigned  
          39.
          Evaluate options to do an early abort of the build Sub-task Open Unassigned  
          40.
          TestSlotZnode concurrency tests are unstable Sub-task Open Unassigned  
          41.
          temp_table_multi_insert_partitioned.q is unstable Sub-task Open Unassigned  
          42.
          schema_evol_par_vec_table_* tests are unstable Sub-task Open Unassigned  
          43.
          authorization_disallow_transform.q is unstable Sub-task Open Unassigned  
          44.
          special_character_in_tabnames_1.q is unstable Sub-task Open Unassigned  
          45.
          Collect ignored tests Sub-task Open Unassigned  

            Activity

              People

              • Assignee:
                kgyrtkirk Zoltan Haindrich
                Reporter:
                kgyrtkirk Zoltan Haindrich
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3.5h
                  3.5h