#960 Mitigate cross device issue Pull request opened by davidmirror-ops Problem

ObstoreParallelReader.download_files

downloads chunks into a

tempfile.TemporaryDirectory()

(defaults to

/tmp

) then moves each completed file to the target directory via

aiofiles.os.replace

(which calls

os.rename

under the hood). When the container has

/tmp

on a separate filesystem from the destination (e.g. a tmpfs mount),

os.rename

fails with:

OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpcdwfp9ad/fast<...>.tar.gz' -> '/root/fast<...>.tar.gz'

This was observed in production at MyHeritage after they bumped to a new base image that mounted /tmp as tmpfs. The error was hidden inside a Python 3.11+ ExceptionGroup which made the surface-level error appear unrelated. Tests Three new unit tests are added (no sandbox / real S3 required): • test_download_files_single_file — happy path with a mocked store • test_download_files_multipart — verifies multi-chunk reassembly • test_download_files_no_cross_device_error — regression test: wraps aiofiles.os.replace to raise EXDEV when the source is outside target_prefix, confirming the fix keeps the staging file on the same device flyteorg/flyte-sdk