<#3320 [Fix] Issue when using FlyteDirectory with ...
# flyte-github
a
#3320 [Fix] Issue when using FlyteDirectory with Elastic Pull request opened by machichima ## Tracking issue Follow up #3313 ## Why are the changes needed? As described in #3313 ## What changes were proposed in this pull request? Ensure that everytime we call
download
, we will get the
current_context
of the current process, to ensure calling the
get_data
synchronously. ## How was this patch tested? Test with script from flytekit import task, workflow, FlyteDirectory, ImageSpec, Resources from flytekitplugins.kfpytorch.task import Elastic, PyTorch DEFAULT_REMOTE_PATH = "s3://my-s3-bucket/test_dir/" image_spec = ImageSpec( registry="localhost:30000", packages=[ "flytekitplugins-kfpytorch[elastic]", "numpy", "torch", ], ) @task( task_config=Elastic(nnodes=1, nproc_per_node=4), container_image=image_spec, requests=Resources(cpu="4", mem="4Gi"), limits=Resources(cpu="4", mem="4Gi"), ) def my_task(dataset: FlyteDirectory): path = dataset.download() print(f"path: {path}", flush=True) assert Path(path).is_file() # will pass now @workflow def wf(): file = FlyteDirectory(path=DEFAULT_REMOTE_PATH) outputs = my_task(dataset=file) ### Setup process ### Screenshots Result [image](https://private-user-images.githubusercontent.com/60069744/480786944-04e5a036-be77-4454-957e-87227d72b721.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYzNjQ5ODksIm5iZiI6MTc1NjM2NDY4OSwicGF0aCI6Ii82MDA2OTc0NC80ODA3ODY5NDQtMDRlNWEwMzYtYmU3Ny00NDU0LTk1N2UtODcyMjdkNzJiNzIxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI4VDA3MDQ0OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTUwMWY3OTVlOGZiYWRmNWM3NDA2NDI5OTkwMDQ3NTEwNzIyZWMyNGJhMmIzMWNhN2NiMjkyOTQ4ZjAzM2YxNWMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.vAfO5Bpwyi0IZl5VTzp8YRnYp_OV1bh5S-1vkvtzcdk) [image](https://private-user-images.githubusercontent.com/60069744/480786972-f7173137-6e8f-4f96-b766-6efef7876d58.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTYzNjQ5ODksIm5iZiI6MTc1NjM2NDY4OSwicGF0aCI6Ii82MDA2OTc0NC80ODA3ODY5NzItZjcxNzMxMzctNmU4Zi00Zjk2LWI3NjYtNmVmZWY3ODc2ZDU4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MjglMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODI4VDA3MDQ0OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTNhNmY0NzAxZTUyY2E0MzI4NWIyZjhhZmFhNjZmNGNiYjg4NjkzNzZmOTEwZDM2ODlmMGJjZDJlMzUyOTkwMzAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.yVl4v9B2lkbjY_6Rm_55Z2vtB8fMcMeJ3bw_bhrSHXg) ## Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. ## Related PRs Follow up #3313 ## Docs link https://bito.ai#summarystart| ### Summary by Bito This pull request introduces a new downloader class for the FlyteDirectory, enhancing the synchronous data retrieval mechanism. It replaces the previous downloader function with a class instance, improving reliability and ensuring correct context utilization during downloads, as per issue #3313. flyteorg/flytekit