https://flyte.org logo
Join Slack
Powered by
Hi Team, My team at Ripple uses Databricks as main...
# flyte-deployment
s
Hi Team, My team at Ripple uses Databricks as main ML Platform. We have been trying to figure out ways to parallelize some of our compute efforts. I am wondering if Databricks Flyte integration would be helpful in this case? I am trying to understand what are the benefits of Flyte/Databricks integration? Our Databricks is deployed on AWS.
k
hey @Shahwar Saleem yes, it can parallelize, manage clusters, you can reuse clusters, run jobs, allow you to cache results and also modify code on demand and run them
it basically makes iterating on Databricks much faster, with imagespec, fast registration and (we recommend starting with Agents)
you can also add jobs that are non databricks that are using k8s containers
Happy to connect and walk over things. There are many folks that are using databricks and Flyte together
s
This is really awesome, this would be my second POC with Flyte. (Good to see you again) Is there any documentation that I can educate myself with before I start meeting 1:1?
k
haha ohh ya, thank you for joining us again 😄 pleasure to work
what type of documentation you are looking for?
the great part of agent is, it all works locally
but, agent is still not fully released - we are very close to it
so we might have to change the api slightly
cc @Kevin Su
s
Okay so I can hook locally running agent with my Databricks environment on AWS?
k
yup, you dont even need sandbox
just 
pip install
and go!
s
I am in research and POC phase of testing integration of Flyte with Databricks. Once, I see prospects I will switch over my team to it.
Okay, let me try a few things out.
Thanks Ketan!
4 Views
Open in Slack
PreviousNext
Powered by