quick-helicopter-88984
08/13/2024, 5:53 PMUSER:AssertionError: error=Expected a directory, but the given uri '/tmp/flyte-9lq_c257/sandbox/local_flytekit/44d6ea3a6ffa4d930e874e79131e4c3fimsop046/tmphwg8139l' is not a directory.
The funny thing is that I don't get this error if I only have one level of map_task nor do I get this if I avoid using map_task and use two layers of dynamic tasks.
Is this expected? I have a few observations:
• The problem gets resolved if my innermost task returns a path String and lets Flyte create FlyteDirectory rather than constructing FlyteDirectory itself
• The path pattens seem to be different when using one level of map_task and two levels ("/tmp/flytearog1v2o/control_plane_metadata/local_flytekit/..." vs "/tmp/flyte-9lq_c257/sandbox/local_flytekit/..."
Here is the code for the inner most task
@task(retries=0) # type: ignore [misc]
def dummy_task(input: str) -> FlyteDirectory:
working_dir = current_context().working_directory
local_dir = Path(tempfile.mkdtemp(dir=working_dir))
out_path = local_dir / "test.txt"
abs_path = out_path.absolute()
with abs_path.open("w") as f:
f.write(input)
# This fails remotely
return FlyteDirectory(str(local_dir.absolute()))
# But this succeeds!
# return str(local_dir.absolute())
quick-helicopter-88984
08/13/2024, 5:58 PMthankful-minister-83577
thankful-minister-83577
thankful-minister-83577
thankful-minister-83577
quick-helicopter-88984
08/13/2024, 6:22 PMquick-helicopter-88984
08/13/2024, 6:23 PM