dry-ability-69144
03/02/2023, 5:42 PM[Errno 2] No such file or directory: '/tmp/flytep4bezuvh/local_flytekit/d8e1f470ac5ed7ac3141ab1b6d83a854/6W2MGqmQ.jpg'
Can anybody help me with this?cool-lifeguard-49380
03/02/2023, 5:57 PMcool-lifeguard-49380
03/02/2023, 5:58 PM.download()
?dry-ability-69144
03/02/2023, 5:59 PM.download()
. Second one I'm just trying to use the .path
to load the file. In this case, using PIL to load imagecool-lifeguard-49380
03/02/2023, 6:00 PM@task
def upload_file() -> FlyteFile:
# Save image to <some_path>
return FlyteFile(path=<some_path>)
@task
def download_file(f: FlyteFile) -> None:
f.download()
dry-ability-69144
03/02/2023, 6:01 PMdry-ability-69144
03/02/2023, 6:05 PMoutput.append(FlyteFile(path=f"gs://{bucket_name}/{img_prefix_path}"))
Then, in second task, I simply download it
@task(cache=True, cache_version="1.0")
def load_file_as_flytefile(file: FlyteFile) -> FlyteFile:
return file.download()
Then, finally, on the last task, I try to read it using PIL
image = Image.open(samples[i].path).convert('RGB')
Sorry, but I cannot give more of the code. I hope this can helpcool-lifeguard-49380
03/02/2023, 6:07 PMcool-lifeguard-49380
03/02/2023, 6:07 PMdry-ability-69144
03/02/2023, 6:07 PMcool-lifeguard-49380
03/02/2023, 6:07 PMcool-lifeguard-49380
03/02/2023, 6:08 PMcool-lifeguard-49380
03/02/2023, 6:08 PM.download()
at the beginning of the 3rd taskcool-lifeguard-49380
03/02/2023, 6:09 PMreturn file.download()
Maybe replace with
file.download()
return file
(The 3rd task will have to download again regardless of this though)dry-ability-69144
03/02/2023, 6:13 PMcool-lifeguard-49380
03/02/2023, 6:14 PMSo, each task must download the file?Yes, this is correct 🙂 Each task runs in its own pod in the kubernetes cluster which could potentially be on a different node than the previous task.
cool-lifeguard-49380
03/02/2023, 6:15 PMFlyteFile
is that it takes care of the serialization and deserialization (to and from a bucket) when passing around files between tasks. Otherwise, you would have to take care of this (de-)serialization yourself.cool-lifeguard-49380
03/02/2023, 6:16 PMdry-ability-69144
03/02/2023, 6:18 PMcool-lifeguard-49380
03/02/2023, 6:21 PMdry-ability-69144
03/02/2023, 6:22 PMcool-lifeguard-49380
03/02/2023, 6:26 PM@task
def downooad_image(f: FlyteFile):
f.download()
return f
… then the image is first downloaded but then uploaded again behind the scenes when calling return f
.dry-ability-69144
03/02/2023, 6:35 PMdry-ability-69144
03/02/2023, 6:36 PMdry-ability-69144
03/02/2023, 6:36 PMcool-lifeguard-49380
03/02/2023, 6:44 PMcool-lifeguard-49380
03/02/2023, 6:45 PMdry-ability-69144
03/02/2023, 6:45 PMcool-lifeguard-49380
03/02/2023, 6:49 PMcool-lifeguard-49380
03/02/2023, 6:49 PMcool-lifeguard-49380
03/02/2023, 6:49 PMdry-ability-69144
03/02/2023, 6:49 PM