Matheus Moreno

    Matheus Moreno

    5 months ago
    Hey, everyone! I'm trying to integrate our MLflow server to our Flyte projects. My current approach consists of passing the MLflow username and password as Secrets to the required tasks, but I was wondering: does anyone else here uses this stack, and if so, are you willing to share a little bit about how it's configured? 🙂
    Lucas Rolim

    Lucas Rolim

    5 months ago
    @Ketan (kumare3) I remember we discussed it in the past and someone was working on this integration. Right?
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    that is true, @Stephen?
    Also, Someone suggested to use the following pattern
    @task
    @mlfow
    def task1():
       ...
    pattern explained here. Someone has done this for clearml as well, but not open sourced it
    it you guys write something, it can be open sourced
    also @Matheus Moreno what is the aim of the integration? is this for logging the runs?
    if so you can also use the workflow decorator - as explained here. BTW, we would love this plugin 😄
    @Lucas Rolim would this help?
    Stephen

    Stephen

    5 months ago
    So for now we haven’t worked on an integration for MLFlow, we use MLFlow but we call the library directly and then we log the different metrics that we want with MLFlow directly. It would be nice to have something like
    @task
    @mlflow
    def task1():
       ...
    as you suggested Ketan. It is still in my backlog but if you peeps wanna work on it feel free to do so, I could help as well I just don’t have the time for now to fully commit to it 😕
    Matheus Moreno

    Matheus Moreno

    5 months ago
    Yes @Ketan (kumare3), that's exactly it. In our stack, we use MLflow to track experiments and save models. Uniting it with Flyte, we could have a place to analyse executions and, consequently, model performance. If I understood correctly, our current approach is very similar to @Stephen's. Does your MLflow server have some sort of authorization layer? If so, how do you make it work? Because right now we're passing Secrets to tasks and setting the required environment variables with a
    configure_mlflow()
    helper function. I think this decorator could work in two ways: a) by simply starting and ending a run, and configuring the connection to the server if needed; or b) same as (a), but also by keeping track of all logs being made inside a function, and batch logging everything after the run is over. Functionality (a) is very trivial, maybe too trivial to become an integration; functionality (b) would be awesome, but I cannot think of a way to implement it without creating an extra layer of work to the end user. Thoughts?
    Ketan (kumare3)

    Ketan (kumare3)

    5 months ago
    Aah I think even if trivial a is awesome
    Other way is - logs and how we can make this a plugin for current context
    It does not have a plugin system today, but should be trivial to implement
    I'll write up an example when at the desk