loud-belgium-4006
05/28/2023, 11:39 PMType of Generic List type is not supported, Transformer for type <class 'tuple'> is restricted currently
So now I'm back to using FlyteDirectory but it's.. awkward since there's no metadata about the files and they're given arbitrary names on the backend unless I explicitly name them somehow.
I'm wondering how folks might go about dealing with this? Do I handle all the metadata via the FlyteFile names? Pass some sort of metadata object along with every FlyteDirectory? Maybe there's a way around the custom dataclass limitation, as that would be the most elegant solution. Thanks for reading! I appreciate any insight.freezing-airport-6809
loud-belgium-4006
05/29/2023, 5:13 PM@task
def get_file_contents(infile: FlyteFile) -> str:
local = Path(infile.path)
print(local.exists())
content = ''
with open(infile, 'r') as in_:
content = in_.read()
return content
This task runs without issue in the current form. However, that print statement will say False
, and if I try to open local
it will fail with a File not found
error.loud-belgium-4006
05/29/2023, 5:16 PMopen
is called. My issue was trying to access FlyteFile.path
directly and passing it to another function, which would fail saying the file wasn't there. I think I falsely assumed this had to do with my custom dataclass.loud-belgium-4006
05/29/2023, 5:18 PMpath
attribute specifies (e.g. /tmp/flytetsfn_0lu/local_flytekit/899733d34a9e428d2773b2c5ffd2914d/hello.txt
) where a FlyteFile will be downloaded to if the download
method is called, not where it actually exists?tall-lock-23197
path
is populated after the file is downloaded. Try setting local
to infile.download()
.loud-belgium-4006
05/30/2023, 5:50 PM