Hello, I’m wondering how involved it would be to e...
# announcements
Hello, I’m wondering how involved it would be to extend the sagemaker plugin to support additional task types like batch transforms and creating/updating endpoint configurations. Is this something that can be done purely in python (flytekit)?
@Andrew Achkar it wont be too much work
it can be done in flytekit (well everything can be), but the backend plugins are very efficient
but, thats a trade-off. I think you should always start with a flytekit only plugin and once happy we can help you make a backend plugin
Also if you like the interface lets share it in the community sync and there are lot of folks who can help with the backend 🙂
Let me know if this helps
also happy to hop on a call to discuss
So with the python only approach would you suggest to use boto3 or sagemaker sdk directly?
Whatever you prefer
cc @Edgar Trujillo / @Andrew Achkar we at Union would love to help communicate. But most of us are our for the next few days. How about 7th / 8th?
@Edgar Trujillo you are absolutely right. But, the problem with sagemaker plugin today is the dependence on the sagemaker operator
We wanted to eventually just migrate away from the Sagemaker operator to use - Flyte’s WEB Api interface, makes it much easier to manage connections without the dependence on the operator, which is very stale
cc @Kevin Su / @Yee
@Ketan (kumare3) Returning to this thread, I think I stumbled into an issue with using Flyte’s Sagemaker training integration that I’m curious if you’ve resolved. As it stands, it doesn’t seem possible to associate a training job with a sagemaker experiment, so it wouldn’t be possible to use SM Studio UI to compare trials or otherwise view training metrics. What kind of pattern makes the most sense to you in terms of having experiment tracking and comparisons on top of training orchestrated by Flyte?
Hey Andrew would you be open for a quick chat
@Ketan (kumare3) yes I am available today to chat
I dmed you