Nizar Hattab
07/02/2023, 12:40 PMCannot convert from /tmp/some_path to typing.Optional[flytekit.types.directory.types.FlyteDirectory]\n\nSYSTEM ERROR! Contact platform administrators."
please note that sometimes it works . does it have to use context path instead ?Kevin Su
07/02/2023, 11:38 PMKetan (kumare3)
Nizar Hattab
07/03/2023, 9:20 AMdef scrape(scraper_input: ScraperMapInput) -> Optional[FlyteDirectory]:
scraper.scrape() # writes multiple files to a output_path directory
remote_dir = os.path.join('<gs://datasets>', output_path)
return FlyteDirectory(path=output_path, remote_directory=remote_dir)
# wf:
result = map_task(scrape, min_success_ratio=0.25)(scraper_input=scraper_inputs)
Kevin Su
07/04/2023, 6:38 AMNizar Hattab
07/04/2023, 10:27 AMAriel Kaspit
07/04/2023, 12:45 PMKevin Su
07/05/2023, 12:32 AM/tmp/some_path
a directory
2. did you see any other errors in the pod?Nizar Hattab
07/05/2023, 10:37 AM