Bug 40249 - gb: parallelize gb-task-check-install-arch for speed
Summary: gb: parallelize gb-task-check-install-arch for speed
Status: NEW
Alias: None
Product: Infrastructure
Classification: Infrastructure
Component: girar (show other bugs)
Version: unspecified
Hardware: all Linux
: P5 enhancement
Assignee: placeholder@altlinux.org
QA Contact: Andrey Cherepanov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-06-21 02:06 MSK by Dmitry V. Levin
Modified: 2021-11-19 03:20 MSK (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dmitry V. Levin 2021-06-21 02:06:55 MSK
When a task built more than just a few binary packages, it makes sense to parallelize gb-task-check-install-arch because the only thing that could be shared between install checks is the cache.
Comment 1 Ivan A. Melnikov 2021-10-13 21:49:04 MSK
I have a rough implementation of parallel install checks that currently works on  mipsel (secondary) and riscv64 girar instances. It uses the same build node and user and runs install checks in parallel using several hasher instances (with different --number). For 4-core Loongson3A I run 5 install check threads, and it gives from 1.6 to 3 times speed up, depending on the task. Also, for trivial tasks where only one binary package is build parallel version may be slower by a second or so.

I'm not posting links to code here mainly because I believe that it will be more effective to write something from scratch then to forward-port & clean up my implementation onto current girar (I'm still using an ancient pre-ga version). Still, I can write smth up on what and how I was doing, if you think it'll be useful.
Comment 2 Dmitry V. Levin 2021-11-19 03:20:56 MSK
(In reply to Dmitry V. Levin from comment #0)
> When a task built more than just a few binary packages, it makes sense to
> parallelize gb-task-check-install-arch because the only thing that could be
> shared between install checks is the cache.

Since the cache is so important for install checks, it would make sense to generate the cache once and subsequently reuse it by parallelized install checks.