[HIVE-13503] [Umbrella] Test sub-system improvements

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Target Version/s: 3.0.0
• Assignee: Unassigned
• Reporter: Siddharth Seth
• Created: 13/Apr/16 20:48
• Updated: 25/Mar/17 21:11

Description

Primarily targeting faster pre-commit builds.

Issue Links

• relates to: HIVE-14547 [Umbrella] Test case fixes / improvements
• relates to: HIVE-14817 Shutdown the SessionManager timeoutChecker thread properly upon shutdown

Sub-Tasks

• Create initial test data once across multiple test runs - TestCliDriver: Patch Available
• Add timeout for individual tests: Patch Available
• Add a timeout for static initialization blocks: Open
• Skip running TestDummy where possibe during precommit builds: Closed
• Investigate speeding up artifact rsyncs in the precommit build: Resolved
• Improved logging for ptest: Closed
• Run clidriver tests from within the qtest dir for the precommit tests: Closed
• Add a generic timeout to non CliDriver tests: Open
• Don't allow any test to run for longer than 60minutes in the ptest setup: Closed
• Reduce the max runtime of a test further: Resolved
• Convert MiniTez tests to MiniLlap tests: Resolved
• Explore surefire parallel testing options: Open
• Run additional tests from the module directory: Resolved
• Add support in ptest to create batches for non qfile tests: Resolved
• Remove tez golden files after HIVE-14502: Resolved
• Enable reuseForks for most modules: Open
• Ptest: Add javadocs/documentation to Ptest code: Open
• Testing: Evaluate and fix overheads in executing a single q test: Resolved
• Sync up configs for MiniTez and MiniLlap: Open
• Minor ptest2 improvements: Resolved
• Minor qtest fixes: Closed
• Add a local cluster for Tez and LLAP: Closed
• Change ptest java language version to 1.7, other version changes and fixes: Resolved
• Have the actual error reported when a q test fails instead of having to go through the logs: Closed
• Ensure directories are cleaned up on test cleanup in QTestUtil: Open
• ptest uses invalid killall command: Resolved
• Add a warning to the jenkins report for slow tests: Open
• Determine unit tests to batch together based on previous run info: Open
• ptest killall command does not work: Resolved
• Reduce number of retries while starting HiveServer for tests: Resolved
• Separate fstype from cluster type in QTestUtil: Closed
• Improve ptest2 build time: Resolved
• Change qtest logging to not redirect all logs to console: Closed
• Add a core cluster type to QTestUtil: Closed
• Reduce the memory used by MiniMr, MiniTez, MiniLlap tests: Closed
• Improve the 'TestClass' did not produce a TEST-*.xml file message: Resolved
• Add an option to skip log collection for successful tests: Resolved
• Reduce the memory requirements for Spark tests: Resolved
• Improve the 'TestClass' did not produce a TEST-*.xml file message to include list of all qfiles in a batch, batch id: Resolved
• ptest - avoid unnecessary cleanup from previous test runs in batch-exec.vm: Resolved
• Test framework integration with findbugs, rat checks etc.: Closed
• Reduce the memory used by unit tests, MiniCliDriver, MiniLlapLocal, MiniSpark: Resolved
• Fix batchSize for TestNegativeCliDriver: Resolved
• Fix ptest2 test failures: Resolved
• Make it possible to run checkstyle for a specific module: Closed
• Make it possible to run findbugs for itest modules as well: Resolved
• Remove YETUS*.sh files after a YETUS release: Closed
• Integreate YETUS to Pre-Commit: Closed
• Run YETUS in Docker container: Open
• Run checkstyle on ptest2 module with proper configuration: Closed
• Fix ASF headers: Closed
• Upgrade checkstyle version to support lambdas: Closed
• Run checkstyle on storage-api module with proper configuration: Closed
• Make sure Yetus check always has a full log: Closed
• Ensure git gc finished in ptest prep phase before copying repo: Closed
• Add findbugs to yetus pre-commit checks: Closed
• Fix findbugs yetus pre-commit checks: Closed