Hi! I am trying to figure out how to use FlyteDirectory that is returned by a task in another task.
My use case is that the first task creates directories with files inside. The task returns a list of FlyteDirectory.
These directories are passed to another task which uses a model on these files.
When running the workflow locally (running the script in pycharm) the workflow is successful and I am able to use the model on these passed directories. But when I register the workflow to the cluster, the task containing the model sees these FlyteDirectories as empty paths (has nothing to predict on).
Has anyone encountered a use case similar to this?