<#7263 perf(devbox): parallelize image preload in ...
# flyte-github
a
#7263 perf(devbox): parallelize image preload in devbox-bundled build Pull request opened by pingsutw Why are the changes needed? The
preload
stage in
docker/devbox-bundled/Dockerfile
pulls, tags, and saves the container images listed in
images/manifest.txt
one at a time. With 8 images in the manifest, cold builds spend most of the stage's wall time waiting on serial registry round-trips even though the operations are independent. What changes were proposed in this pull request? Rewrite
docker/devbox-bundled/images/preload
to run `podman pull`/`tag`/`save` for every manifest entry concurrently and then
wait
for them all: • All entries launch in parallel as background jobs; the script exits non-zero if any job fails. • Each image's stdout/stderr is buffered to a per-image
.log
file and only emitted on failure, so successful runs stay readable and failed ones clearly identify which image broke. • Behavior and output artifacts (per-image
.tar
files under
images/
) are unchanged. Net effect: preload runtime drops from roughly the sum of per-image pull times to roughly the max. How was this patch tested? •
sh -n docker/devbox-bundled/images/preload
— syntax check passes. • Local
make build
in
docker/devbox-bundled/
completes successfully with a visibly shorter preload stage. Labels changed Check all the applicable boxes • All commits are signed-off. •
main
• #6583 • perf(devbox): parallelize image preload in devbox-bundled build 👈 flyteorg/flyte