Pryce05/28/2023, 11:39 PM
So now I'm back to using FlyteDirectory but it's.. awkward since there's no metadata about the files and they're given arbitrary names on the backend unless I explicitly name them somehow. I'm wondering how folks might go about dealing with this? Do I handle all the metadata via the FlyteFile names? Pass some sort of metadata object along with every FlyteDirectory? Maybe there's a way around the custom dataclass limitation, as that would be the most elegant solution. Thanks for reading! I appreciate any insight.
Type of Generic List type is not supported, Transformer for type <class 'tuple'> is restricted currently
Pryce05/29/2023, 5:13 PM
This task runs without issue in the current form. However, that print statement will say
@task def get_file_contents(infile: FlyteFile) -> str: local = Path(infile.path) print(local.exists()) content = '' with open(infile, 'r') as in_: content = in_.read() return content
, and if I try to open
it will fail with a
File not found
is called. My issue was trying to access
directly and passing it to another function, which would fail saying the file wasn't there. I think I falsely assumed this had to do with my custom dataclass.
attribute specifies (e.g.
) where a FlyteFile will be downloaded to if the
method is called, not where it actually exists?
is populated after the file is downloaded. Try setting
Pryce05/30/2023, 5:50 PM