Has anyone thought about TFX and Flyte

Question

Ketan (kumare3) · Answer

< Mike Seid> we did

Ketan (kumare3) · Answer

just do not have the bandwidth or folks to do it

Ketan (kumare3) · Answer

we would love to help if someone wants to do it

Ketan (kumare3) · Answer

TFX should be simple to fit into flytekit as an extension

Mike Seid · Answer

got it super interesting Yeah seems pretty straight forward I ll let you know if we decide to pursue it further If you had to ballpark how hard long it would take to implement what would you say I hate typical man month but not sure what other unit is better

Ketan (kumare3) · Answer

haha

Ketan (kumare3) · Answer

do you have a few minutes

Ketan (kumare3) · Answer

maybe we can walk through the execution model and then guess

Ketan (kumare3) · Answer

from what I see it seems straightforward <https github com tensorflow tfx blob master tfx orchestration pipeline py L195|here> I think to get a prototype working for someone who knows tfx working with a flyte person might be a week or so less maybe

Ketan (kumare3) · Answer

and then a full fledged support might be close to a month

Ketan (kumare3) · Answer

lets pad it and say a little more

Kevin Su · Answer

Have some experience on TFX and love to help integrate TFX

Ketan (kumare3) · Answer

< Kevin Su> you are a rockstar then you know both Flytekit and TFX

Mike Seid · Answer

Sounds interesting Let me float some ideas to my team and see what the uptake is I ll get back to you Appreciate the quick informatoin