<#6148 [BUG] Fail to walk through a FlyteDirectory...
# flytekit
c
#6148 [BUG] Fail to walk through a FlyteDirectory through os.listdir Issue created by JiangJiaWei1103 Describe the bug Currently,
FlyteDirectory
doesn't support
os.listdir
with a remote S3 path (fortunately, a local path works!). The error is shown as follows: Screenshot 2025-01-08 at 9 10 58 PM Though
FlyteDirectory
provides
listdir
method itself, I just think
os.listdir
could provide a better UX. Expected behavior A
FlyteDirectory
with a remote S3 path can be walked through by
os.listdir
, as commented here Additional context to reproduce Create a
demo
dir on minio and add a text file first. Then, run the following script: import os from flytekit import task from flytekit.types.directory import FlyteDirectory S3_DIR = "s3://my-s3-bucket/demo @task def create_fd() -> FlyteDirectory: fd = FlyteDirectory(path=S3_DIR) entities = os.listdir(fd) for e in entities: # Suppose there's no subdir with open(os.path.join(S3_DIR, e), "r") as f: print(f.read()) return fd if name == "__main__": fd = create_fd() print(fd) Screenshots No response Are you sure this issue hasn't been raised already? • Yes Have you read the Code of Conduct? • Yes flyteorg/flyte