hi folks! looking for some documentation on ray in...
# flytekitd
Dylan Wilder
03/17/2023, 7:08 PMhi folks! looking for some documentation on ray integration - esp as it pertains to persistent clusters. can't find my last question on this since it was more than 90d ago. even the original RFC would suffice, the public docs don't go into detail on limitations/architecture
k
Ketan (kumare3)
03/17/2023, 8:06 PM
we do not support persistent clusters today
d
Dylan Wilder
03/17/2023, 8:07 PMany timeline on that?
k
Ketan (kumare3)
03/17/2023, 8:07 PM
Ketan (kumare3)
03/17/2023, 8:07 PM
nothing we can commit - please contribute 🙂
Ketan (kumare3)
03/17/2023, 8:07 PM
TBH no one asked for it
Ketan (kumare3)
03/17/2023, 8:07 PM
even folks at spotify found corruption issues
d
Dylan Wilder
03/17/2023, 8:08 PMlast i heard it was a blocker on ray's side
Dylan Wilder
03/17/2023, 8:08 PMyea could use more info on that. is there a ticket out there?
k
Ketan (kumare3)
03/17/2023, 8:08 PM
ya ray would get corrupted
Ketan (kumare3)
03/17/2023, 8:08 PM
i would not know
d
Dylan Wilder
03/17/2023, 8:08 PMsomeone on our side? 🙂
k
Ketan (kumare3)
03/17/2023, 8:08 PM
sorry @Dylan Wilder / but, this was tried by folks at spotify and did not really tell us - as this is a big deal
Ketan (kumare3)
03/17/2023, 8:08 PM
yup 🙂
d
Dylan Wilder
03/17/2023, 8:10 PMcool. will check there. but from your perspective, persistent resources across tasks do work? or youre waiting on implementation
k
Ketan (kumare3)
03/17/2023, 9:04 PM
no there is no implementation in flyte today
Ketan (kumare3)
03/17/2023, 9:04 PM
we need better requirements
150 Views