Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16604

Parallelise docker container runs for tests in ci-cassandra.a.o

    XMLWordPrintableJSON

    Details

      Description

      This was raised on the dev ML, where the consensus was to remove it: https://lists.apache.org/thread.html/r1ca3c72b90fa6c57c1cb7dcd02a44221dcca991fe7392abd8c29fe95%40%3Cdev.cassandra.apache.org%3E

      The idea is to then replace ant test parallelism with docker container parallelism.

      PoC patch:
      https://github.com/apache/cassandra-builds/compare/trunk...thelastpickle:mck/16587-2/trunk

      This is just a quick PoC, aimed at the ci-cassandra agents that have
      4 cores and 16gb ram available to each executor, but I imagine instead
      something that spawns a number of containers based on system
      resources, like we currently do with get-cores and get-mem.

      Also worth noting the overhead here, compared with the ant parallelism approach, docker
      builds everything in each container from scratch, but this too can be
      improved easily enough.

      Cleaning up any remnant `-Dtest.runners=` options is also part of this ticket.

        Attachments

        1. image-2021-04-28-09-41-20-310.png
          442 kB
          Tomasz Lasica

          Issue Links

            Activity

              People

              • Assignee:
                mck Michael Semb Wever
                Reporter:
                mck Michael Semb Wever
                Authors:
                Michael Semb Wever
                Reviewers:
                Tomasz Lasica
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: