Hi! I saw on the website that experiment tracking and data lineage is on the way. Is there anywhere this progress can be tracked? I’m working at a company that’s just making a switch from determined.ai to Flyte for orchestration. However, this now means we are missing three important pieces which are experiment tracking/model registry/data versioning. Wondering if union.ml could be a good fit if it’s not too far away on the horizon?

Hi Martin welcome to the community

Hi @freezing-airport-6809! Thanks for the quick reply. I was referring to this from your website.

So we are working with wandb and hopefully mlflow too

hi Martin! So for unionml we’re planning to build integrations for MLFlow for experiment tracking and potetially model registry (although Flyte itself is basically a model registry), you can check out the roadmap here: https://github.com/orgs/unionai-oss/projects/1/views/4

yeah we have an ongoing project to send events to datahub, but not sure if that would be sufficient for your usecase. would like to learn more about it.

Fwiw, we use mlflow to track experiment metrics as well. Datahub operates at a different abstraction level. It allows us to link tasks to datasets (tables for example) and models (on a ”app level”, not individual model versions).

I think for experiment tracking we’re planning on relying on integrations with other libraries in the ecosystem (e.g. mlflow)

That makes sense @broad-monitor-993. For the tracking, we mostly need rudimentary features (eg graphs). For the model registry, we want a centralized thin abstraction layer in front of a blob storage (eg s3) that can reference arbitrary models and metadata. MLFlow suits our needs pretty well here. When it comes to data lineage, we’re currently using DVC but I’m currently looking for alternatives as using git to track revisions isn’t great when domain experts need to be able to manipulate the datasets (fix incorrect labels etc).

@faint-monitor-96441 do you think we could have a separate quick chat just about data versioning?

Sure, maybe tomorrow? I’m at CET, when’s a good time for you?

would love to hear how you’re currently using it and any other usecases you might have.

1pm Seattle time is a bit late for me in Stockholm

ooh cet

That works!

sweeet, let me send an invite

Yee is getting used to those early morning meets? 😅 (I’m in helsinki myself)

not sure if you want to join niels/fredrik, but yeah really want to understand the data versioning story here

Can’t tomorrow, unfortunately, but I trust you’ll take good notes 👍

sweet will do.