[HIVE-19571] Ability to run multiple pre-commit jobs on a ptest server

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Testing Infrastructure
• Assignee: Unassigned
• Reporter: Sahil Takiar
• Created: 16/May/18 12:46
• Updated: 27/Feb/24 22:24

Description

I've been taking a look at the Disk, Network, and CPU usage of the GCE instances we run ptest on, and it doesn't look like we are fully utilizing the machines. The resource usage is very up and down.

During each ptest execution, there is a large chunk of time (~20 min) where its just the Jenkins job that is doing any work (checking out github repos, building code, figuring out test batches, etc.). During this time, the ptest nodes are mostly idle - the CPU and Disk I/O are almost zero.

Even when ptest is running, I think some of resources are under-utilized. Network and disk resource spike at the beginning of the job, probably because ptest is distributing resources to each machine, each slave is downloading jars, etc. However, after that, when the actual tests run, there is almost 0 network activity (which makes sense since tests runs on a single node). For disk usage, there is activity, but not nearly as high as when the setup phase was occuring. CPU usage fluctuates between 40-80%.