Hi !
I am working on a task that is gonna be run i...
# flyte-supporti
important-hamburger-34837
07/02/2023, 12:40 PMHi !
I am working on a task that is gonna be run in map task config, each instance will generate few files and will store them in /tmp/some_path, it returns a flyte directory from that path. however I keep getting this error:
Copy code
Cannot convert from /tmp/some_path to typing.Optional[flytekit.types.directory.types.FlyteDirectory]\n\nSYSTEM ERROR! Contact platform administrators."g
glamorous-carpet-83516
07/02/2023, 11:38 PMstring is fine. it’s wired. could you share the code snippet. happy to investigate at it
f
freezing-airport-6809
07/03/2023, 5:16 AM
I think this is the optional bug that I fixed recently? Version will be important
i
important-hamburger-34837
07/03/2023, 9:20 AM Copy code
def scrape(scraper_input: ScraperMapInput) -> Optional[FlyteDirectory]:
scraper.scrape() # writes multiple files to a output_path directory
remote_dir = os.path.join('<gs://datasets>', output_path)
return FlyteDirectory(path=output_path, remote_directory=remote_dir)
# wf:
result = map_task(scrape, min_success_ratio=0.25)(scraper_input=scraper_inputs)important-hamburger-34837
07/03/2023, 9:22 AM@freezing-airport-6809 flytekit 1.5.0
g
glamorous-carpet-83516
07/04/2023, 6:38 AMcould you try flytekit 1.7.1 instead
i
important-hamburger-34837
07/04/2023, 10:27 AMi used 1.7.0 as its on pip index(latest). didnt work
e
elegant-toddler-67101
07/04/2023, 12:45 PMBtw (I work with @important-hamburger-34837) - we use Flyte helm chart on GKE cluster version 1.6.2, I wondered maybe this can be related?
elegant-toddler-67101
07/04/2023, 2:16 PMWe upgraded the chart to 1.7.0 and still getting the error…
elegant-toddler-67101
07/04/2023, 2:17 PMDo you have any idea why it happens? It is a major bug and it blocks us from continue the work with Flyte
g
glamorous-carpet-83516
07/05/2023, 12:32 AMlooking
glamorous-carpet-83516
07/05/2023, 1:34 AMfew questions
1. is
/tmp/some_pathi
important-hamburger-34837
07/05/2023, 10:37 AM1. yes
2. turned out its because of gcs account was not permitted to delete objects. error was not showing up until I removed Optional datatype
important-hamburger-34837
07/05/2023, 10:37 AMThanks
153 Views