Hey, everyone! I'm trying to integrate our MLflow ...
# announcements
m
Hey, everyone! I'm trying to integrate our MLflow server to our Flyte projects. My current approach consists of passing the MLflow username and password as Secrets to the required tasks, but I was wondering: does anyone else here uses this stack, and if so, are you willing to share a little bit about how it's configured? 🙂
l
@Ketan (kumare3) I remember we discussed it in the past and someone was working on this integration. Right?
k
that is true, @Stephen?
Also, Someone suggested to use the following pattern
Copy code
@task
@mlfow
def task1():
   ...
pattern explained here. Someone has done this for clearml as well, but not open sourced it
it you guys write something, it can be open sourced
also @Matheus Moreno what is the aim of the integration? is this for logging the runs?
if so you can also use the workflow decorator - as explained here. BTW, we would love this plugin 😄
@Lucas Rolim would this help?
s
So for now we haven’t worked on an integration for MLFlow, we use MLFlow but we call the library directly and then we log the different metrics that we want with MLFlow directly. It would be nice to have something like
Copy code
@task
@mlflow
def task1():
   ...
as you suggested Ketan. It is still in my backlog but if you peeps wanna work on it feel free to do so, I could help as well I just don’t have the time for now to fully commit to it 😕
m
Yes @Ketan (kumare3), that's exactly it. In our stack, we use MLflow to track experiments and save models. Uniting it with Flyte, we could have a place to analyse executions and, consequently, model performance. If I understood correctly, our current approach is very similar to @Stephen's. Does your MLflow server have some sort of authorization layer? If so, how do you make it work? Because right now we're passing Secrets to tasks and setting the required environment variables with a
configure_mlflow()
helper function. I think this decorator could work in two ways: a) by simply starting and ending a run, and configuring the connection to the server if needed; or b) same as (a), but also by keeping track of all logs being made inside a function, and batch logging everything after the run is over. Functionality (a) is very trivial, maybe too trivial to become an integration; functionality (b) would be awesome, but I cannot think of a way to implement it without creating an extra layer of work to the end user. Thoughts?
k
Aah I think even if trivial a is awesome
Other way is - logs and how we can make this a plugin for current context
It does not have a plugin system today, but should be trivial to implement
I'll write up an example when at the desk
154 Views