Hi !
I am working on a task that is gonna be run i...
# ask-the-communityn
Nizar Hattab
07/02/2023, 12:40 PMHi !
I am working on a task that is gonna be run in map task config, each instance will generate few files and will store them in /tmp/some_path, it returns a flyte directory from that path. however I keep getting this error:
Copy code
Cannot convert from /tmp/some_path to typing.Optional[flytekit.types.directory.types.FlyteDirectory]\n\nSYSTEM ERROR! Contact platform administrators."k
Kevin Su
07/02/2023, 11:38 PMstring is fine. it’s wired. could you share the code snippet. happy to investigate at it
k
Ketan (kumare3)
07/03/2023, 5:16 AM
I think this is the optional bug that I fixed recently? Version will be important
n
Nizar Hattab
07/03/2023, 9:20 AM Copy code
def scrape(scraper_input: ScraperMapInput) -> Optional[FlyteDirectory]:
scraper.scrape() # writes multiple files to a output_path directory
remote_dir = os.path.join('<gs://datasets>', output_path)
return FlyteDirectory(path=output_path, remote_directory=remote_dir)
# wf:
result = map_task(scrape, min_success_ratio=0.25)(scraper_input=scraper_inputs)Nizar Hattab
07/03/2023, 9:22 AM@Ketan (kumare3) flytekit 1.5.0
k
Kevin Su
07/04/2023, 6:38 AMcould you try flytekit 1.7.1 instead
n
Nizar Hattab
07/04/2023, 10:27 AMi used 1.7.0 as its on pip index(latest). didnt work
a
Ariel Kaspit
07/04/2023, 12:45 PMBtw (I work with @Nizar Hattab) - we use Flyte helm chart on GKE cluster version 1.6.2, I wondered maybe this can be related?
Ariel Kaspit
07/04/2023, 2:16 PMWe upgraded the chart to 1.7.0 and still getting the error…
Ariel Kaspit
07/04/2023, 2:17 PMDo you have any idea why it happens? It is a major bug and it blocks us from continue the work with Flyte
k
Kevin Su
07/05/2023, 12:32 AMlooking
Kevin Su
07/05/2023, 1:34 AMfew questions
1. is
/tmp/some_pathn
Nizar Hattab
07/05/2023, 10:37 AM1. yes
2. turned out its because of gcs account was not permitted to delete objects. error was not showing up until I removed Optional datatype
Nizar Hattab
07/05/2023, 10:37 AMThanks
150 Views