@calm-zoo-68637 / @icy-tent-98067 maybe we should automatically hash FlytePickle, as we make it cacheable. Cc @glamorous-carpet-83516 / @high-accountant-32689 We could generate the hash and also add object type to it for extra correctness?

I can imagine a lot of use cases where that would be very useful. IMHO cacheability is the most valuable benefit of workflow orchestration My project is quite content with just using dataclasses and Flyte files / dirs, however. I don’t use FlytePickle whatsoever.

Good that’s what I would love for everyone

Oh yeah. Artifact management is the bane of every DS project. Sure thing, happy to test the functionality.

Ohh did you see the union.ai website

Regarding the serverless offering?

No about artifact and auto triggers/ reactive workflow

Oh. Negative I haven't seen it yet.

Woah. Woah. That is crazy. I enjoyed the managed artifacts with arbitrary key-value attributes from KFP v2. I was just expected something comparable. But querying a managed artifact from within a task using those same attributes? That is very pretty. It would make for a much cleaner way to build out complex MLOps workflows.

@icy-tent-98067 please share https://www.union.ai/blog-post/move-fast-and-dont-break-things-introducing-artifacts-lineage-and-reactive-workflows

The ideas of contracts and lineage are also very pretty. Reading this article made me realize that I’ve internally normalized unnecessarily over engineered solutions from my previous experiences. For example, I spent a whole day working out a means to create a versioned label for my models that I internally build in order to be compatible with artifact caching (multiple finetuned models originating from any given cached pretrained model). I use this versioned label for artifact storage, training logs, and model deployment. This is pretty standard so I’ve grown accustomed to handling it manually. However, my over engineered solution is just yet another thing to manually manage and keep in mind. I’ll upgrade my system to be better utilize these managed artifacts and enjoy the benefits. I absolutely agree with the premise of this post. Passing around artifacts as URIs is bad practice. Querying them via partitions is beautiful. Tracking lineage allows for accountability. The sum of all of the above allows for better MLOps at scale. It is very well written. Let me integrate it this weekend and better collect my thoughts.

It’s only part of Union