#790 feat: consolidate image and bundle caches into LocalDB Pull request opened by pingsutw Summary • Adds a persistent SQLite disk cache for image existence checks and code bundle uploads, consolidated into the shared

LocalDB

database (

~/.flyte/local-cache/cache.db

) • Image cache: after the first successful remote registry check (~4s for

docker manifest inspect

), subsequent runs hit the local SQLite cache (~0ms). Cache key is a SHA-256 of repository, tag, and architecture. • Bundle cache: if

list_files_to_bundle

returns the same digest as a previous run, the upload is skipped entirely and the cached remote path is reused. The cache check happens before

create_bundle

so tar creation is also avoided on cache hit. • Both caches have a 30-day TTL with probabilistic pruning (~5% of reads) to avoid stale entries without adding overhead on every access. •

PersistentCacheImageChecker

is inserted as the first checker in the

DockerImageBuilder

checker chain, before network-based checkers. Key changes •

_persistence/_db.py

: Added

image_cache

and

bundle_cache

table DDLs to

LocalDB

, with a unified

_ALL_TABLE_DDLS

list for DRY initialization across sync and async paths •

image_builder.py

: Removed standalone

~/.flyte/cache/images.db

— reads/writes now go through

LocalDB.get_sync()

with

_write_lock

for thread safety •

bundle.py

: Removed standalone

~/.flyte/cache/bundles.db

— same consolidation pattern; uses lazy import of

LocalDB

to avoid circular imports •

_task_cache.py

: Added missing

_write_lock

to

_set_sync

and

clear

for consistency with all other sync write paths Test plan •

test_cached

— verifies persistent cache is checked first and other checkers are skipped •

test_persistent_cache_write_and_read

— verifies write/read round-trip and arch isolation via SQLite • All existing image builder tests pass • Manual test:

LOG_LEVEL=10 python examples/basics/hello.py

— first run writes cache, second run shows "found in persistent cache" in ~0ms flyteorg/flyte-sdk