Hi, I'm trying to use flyte with mlflow. I have th...
# ask-the-community
k
Hi, I'm trying to use flyte with mlflow. I have the flyte sandbox up and running and an mlflow docker container running in the sandbox. I created a user defined bridge network [from here] and can connect the mlflow docker container to it. Is there anyway to pass the
--net
flag to the flyte execution so that it can network to other containers using their name(e.g.
https://<dockercontainername>:5000
) instead of the ip? Is there a better way of setting up mlflow instead of it having its own docker container that Is manually started using
docker run
?
s
cc: @Ketan (kumare3)
Would you be able to share the MLFlow example you worked on?
k
@Kyle B we have an mlflow plugin in progress
Would love your insights. Also cc @Katrina P
k
I Don't think I can actually share code, but the setup I have is very simple. It's just
flytectl sandbox exec -- docker network create mynet
then i can start up the mlflow docker container using
flytectl sandbox exec -- docker run --net mynet -it <mlflow docker name>
, Where I run the mlflow server and expose port 5000. I just want to pass that
--net
parameter to
flytectl create execution ...
somehow so i can communicate to the mlflow server by name.
k
you could install mlflow server as deployment in the k8s in the sandbox, so the task pod can connect to the mlflow server through kubedns
n
+1 to integrating with MLFlow server… having played around with the UI I think it’s already doing a lot of work re: design work, understanding metrics across trials, etc.
k
I'm sorry, I'm very new to containerization, would you mind linking me to a resource describing what K8s is and how I'd edit it for flyte?
k
Here is am example to deploy mlflow on k8s
k
Thank you
k
Here is the mlflow plugin we’re working on. Feel free to leave a comments, we’d like to know your thought. Currently this plugin write the params and metric to flyte deck, we want to write metrics to the mlflow server as well.
k
In line with what Kevin mentioned earlier, we have MLFlow set up in its own separate namespace in the same kubernetes cluster and we have run some demo workflows that are able to connect and register models with mlflow this way
119 Views