Repository navigation
Conversation
No longer pass a pessimistic -j2 option by default. Instead, rely on --fast-ci/--slow-ci which uses -j0 by default: use os.process_cpu_count() or os.cpu_count() to decide the number of jobs to run.
|
--fast-ci and --slow-ci uses -j0 by default: https://github.com/python/cpython/blob/main/Lib/test/libregrtest/cmdline.py#L480-L483 When -j0 is used, libregrtest uses os.process_cpu_count() was added to Python 3.13 and is different than os.cpu_count(): it takes in account CPU affinity: https://docs.python.org/dev/library/os.html#os.process_cpu_count Example: |
|
|
||
| def create_test_opts(self, branch, worker): | ||
| testopts = [*self.testFlags, *get_j_opts(worker, 2)] | ||
| testopts = [*self.testFlags, *get_j_opts(worker)] |
There was a problem hiding this comment.
This will reduce parallelism for UnixInstalledBuild which invokes regrtest directly with no --slow-ci.
Also, it may be a little risky for sanitisers to have no cap...
There was a problem hiding this comment.
Oh, I didn't notice that UnixInstalledBuild doesn't use --slow-ci. That's bad! I added --slow-ci to make sure that all buildbots run the test suite the same way: pass same options.
I wrote #815 to add --slow-ci to UnixInstalledBuild and ValgrindBuild.
|
Should be check with worker owners if it's OK to increase the load? |
According to #810 (comment), many builders already configure the number of worker processes. But I'm not sure how to read these tables. @hugovk: do these tables include buildbot workers which don't configure the number of worker processes? For example, I don't see angelico-debian-amd64 in these tables. If I check a angelico-debian-amd64 job: it has 2 CPUs and uses "2 worker processes". With this change, it will use 4 worker processes. |
No longer pass a pessimistic -j2 option by default. Instead, rely on --fast-ci/--slow-ci which uses -j0 by default: use os.process_cpu_count() or os.cpu_count() to decide the number of jobs to run.