Hi! I am trying to figure out how to use FlyteDirectory that is returned by a task in another task.
My use case is that the first task creates directories with files inside. The task returns a list of FlyteDirectory.
These directories are passed to another task which uses a model on these files.
When running the workflow locally (running the script in pycharm) the workflow is successful and I am able to use the model on these passed directories. But when I register the workflow to the cluster, the task containing the model sees these FlyteDirectories as empty paths (has nothing to predict on).
Has anyone encountered a use case similar to this?
01/11/2024, 3:15 PM
This should not happen can you provide a code snippet
01/11/2024, 4:03 PM
It seems I was looking at things the wrong way, it works now 🙂