Hi there, what would be involved in being able to pass information through the type system to the data persistence layer to apply tags to objects that get stored in blob stores via flytekit? Eg. say I'd like to output a FlyteFile of some type and give it a tag so when it gets stored in s3 (or gcp) it will have that tag (eg. it helps associate the object with a project or department). At the moment, it seems the data persistence layer interface really doesn't know anything except paths of objects to/from destinations. Would it be feasible to plumb tags through to the persistence layer via type annotations (for backends that support tags)?

The data persistence layer is getting completely overhauled

Sure. Ideally we'd be able to construct a FlyteFile or FlyteDirectory object ourselves to return and pass in an optional

tags

that would get populated with the object(s) when they are stored to the backend storage layer like s3. Being able to set tags on objects would help for purposes like record keeping (eg. which project created these objects) and might assist with other things like automated deletion using lifecycle policies that filter based on tag keys/values. I think I see a way to do it in flytekit as-is, so I'm curious if the persistence layer is being overhauled whether it would still be compatible or perhaps even easier to add such a capability.

were you thinking of using `typing.Annotated``?

I had initially thought of using the typing annotations for this, but after looking into it I think it would be clearer to pass in

tags

as an optional to the

FlyteFile

(or

FlyteDirectory

) constructor. This means you only get tags if you return a

FlyteFile

and populate this optional kwarg - which preserves the simpler case in backwards compatible way, but allows for more advanced scenarios to opt-in. This makes it possible to populate tag values with runtime information as well.