Hi ! I am working on a task that is gonna be run i...
# ask-the-community
n
Hi ! I am working on a task that is gonna be run in map task config, each instance will generate few files and will store them in /tmp/some_path, it returns a flyte directory from that path. however I keep getting this error:
Copy code
Cannot convert from /tmp/some_path to typing.Optional[flytekit.types.directory.types.FlyteDirectory]\n\nSYSTEM ERROR! Contact platform administrators."
please note that sometimes it works . does it have to use context path instead ?
k
string is fine. it’s wired. could you share the code snippet. happy to investigate at it
k
I think this is the optional bug that I fixed recently? Version will be important
n
Copy code
def scrape(scraper_input: ScraperMapInput) -> Optional[FlyteDirectory]: 

scraper.scrape() # writes multiple files to a output_path directory   
remote_dir = os.path.join('<gs://datasets>', output_path)
    return FlyteDirectory(path=output_path, remote_directory=remote_dir)

# wf: 
result = map_task(scrape, min_success_ratio=0.25)(scraper_input=scraper_inputs)
@Ketan (kumare3) flytekit 1.5.0
k
could you try flytekit 1.7.1 instead
n
i used 1.7.0 as its on pip index(latest). didnt work
a
Btw (I work with @Nizar Hattab) - we use Flyte helm chart on GKE cluster version 1.6.2, I wondered maybe this can be related?
We upgraded the chart to 1.7.0 and still getting the error…
Do you have any idea why it happens? It is a major bug and it blocks us from continue the work with Flyte
k
looking
few questions 1. is
/tmp/some_path
a directory 2. did you see any other errors in the pod?
n
1. yes 2. turned out its because of gcs account was not permitted to delete objects. error was not showing up until I removed Optional datatype
Thanks
117 Views